============================= test session starts ============================== platform linux -- Python 3.7.5, pytest-5.4.3, py-1.8.1, pluggy-0.13.1 rootdir: /home/jenkins/mindspore/testcases/testcases/tests/st/msrun, inifile: /home/jenkins/sault/virtual_test/virtualenv_009/sault/config/pytest.ini plugins: ordering-0.6, anyio-3.7.1, timeout-2.1.0, xdist-1.32.0, forked-1.1.3 collected 1 item test_entry_msrun.py [INFO] ATRACE(163896,python3.7):2024-01-10-11:36:55.738.032 [trace_attr.c:105](tid:163896) platform is 1. [INFO] ATRACE(163896,python3.7):2024-01-10-11:36:55.738.185 [trace_recorder.c:114](tid:163896) use root path: /home/jenkins/ascend/atrace [INFO] ATRACE(163896,python3.7):2024-01-10-11:36:55.738.213 [trace_signal.c:133](tid:163896) register signal handler for signo 2 succeed. [INFO] ATRACE(163896,python3.7):2024-01-10-11:36:55.738.225 [trace_signal.c:133](tid:163896) register signal handler for signo 15 succeed. [INFO] CORE(163896,ffff93db5010,python3.7):2024-01-10-11:36:56.146.371 [mindspore/core/utils/ms_context.cc:225] set_backend_policy] ms set context backend policy:ge [INFO] RUNTIME(163896,python3.7):2024-01-10-11:36:56.146.556 [runtime.cc:1159] 163896 GetAicoreNumByLevel: workingDev_=0 [INFO] RUNTIME(163896,python3.7):2024-01-10-11:36:56.146.598 [runtime.cc:4719] 163896 GetVisibleDevices: ASCEND_RT_VISIBLE_DEVICES param was not set [INFO] DEVICE(163896,ffff93db5010,python3.7):2024-01-10-11:36:56.147.389 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:580] SetContextSocVersion] The soc version :Ascend910A [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:36:56.191.863 [mindspore/ccsrc/pybind_api/ir/log_adapter_py.h:34] PyExceptionInitializer] Set exception handler [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:36:56.203.009 [mindspore/ccsrc/pipeline/jit/ps/init.cc:179] pybind11_init__c_expression] Start GraphExecutorPy... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:36:56.203.661 [mindspore/ccsrc/pipeline/jit/ps/init.cc:271] pybind11_init__c_expression] Start ParallelContext... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:36:56.204.255 [mindspore/ccsrc/pipeline/jit/ps/init.cc:379] pybind11_init__c_expression] Start CostModelContext... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:36:56.204.741 [mindspore/ccsrc/pipeline/jit/ps/init.cc:481] pybind11_init__c_expression] Start OffloadContext... [INFO] DEVICE(163896,ffff93db5010,python3.7):2024-01-10-11:36:56.206.704 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:580] SetContextSocVersion] The soc version :Ascend910A [INFO] CORE(163896,ffff93db5010,python3.7):2024-01-10-11:36:58.368.962 [mindspore/core/utils/ms_context.cc:362] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] PARALLEL(163896,ffff93db5010,python3.7):2024-01-10-11:36:58.369.066 [mindspore/ccsrc/frontend/parallel/costmodel_context.cc:30] GetInstance] Create costmodel_context [INFO] ME(163896:281473162366992,MainProcess):2024-01-10-11:36:58.369.780 [mindspore/run_check/_check_version.py:544] Setting the env `PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python` to prevent memory overflow during save or load checkpoint file. [WARNING] ME(163896:281473162366992,MainProcess):2024-01-10-11:37:00.578.070 [mindspore/parallel/cluster/process_entity/_api.py:186] Start worker process with rank id:0, log file:worker_0.log [WARNING] ME(163896:281473162366992,MainProcess):2024-01-10-11:37:00.649.432 [mindspore/parallel/cluster/process_entity/_api.py:186] Start worker process with rank id:1, log file:worker_1.log [WARNING] ME(163896:281473162366992,MainProcess):2024-01-10-11:37:00.724.892 [mindspore/parallel/cluster/process_entity/_api.py:186] Start worker process with rank id:2, log file:worker_2.log [WARNING] ME(163896:281473162366992,MainProcess):2024-01-10-11:37:00.802.515 [mindspore/parallel/cluster/process_entity/_api.py:186] Start worker process with rank id:3, log file:worker_3.log [WARNING] ME(163896:281473162366992,MainProcess):2024-01-10-11:37:00.802.904 [mindspore/parallel/cluster/process_entity/_api.py:158] Distributed job is spawned. Waiting all processes to exit... [ERROR] ME(163896:281473162366992,MainProcess):2024-01-10-11:41:10.203.348 [mindspore/parallel/cluster/process_entity/_api.py:212] Worker process 164039 exit with exception. [ERROR] ME(163896:281473162366992,MainProcess):2024-01-10-11:41:11.658.922 [mindspore/parallel/cluster/process_entity/_api.py:212] Worker process 164043 exit with exception. [ERROR] ME(163896:281473162366992,MainProcess):2024-01-10-11:41:11.659.214 [mindspore/parallel/cluster/process_entity/_api.py:212] Worker process 164046 exit with exception. [ERROR] ME(163896:281473162366992,MainProcess):2024-01-10-11:41:13.261.70 [mindspore/parallel/cluster/process_entity/_api.py:220] Analyzing exception log... [INFO] ATRACE(164038,python):2024-01-10-11:37:01.922.564 [trace_attr.c:105](tid:164038) platform is 1. [INFO] ATRACE(164038,python):2024-01-10-11:37:01.922.731 [trace_recorder.c:114](tid:164038) use root path: /home/jenkins/ascend/atrace [INFO] ATRACE(164038,python):2024-01-10-11:37:01.922.757 [trace_signal.c:133](tid:164038) register signal handler for signo 2 succeed. [INFO] ATRACE(164038,python):2024-01-10-11:37:01.922.769 [trace_signal.c:133](tid:164038) register signal handler for signo 15 succeed. [INFO] CORE(164038,ffff87d23440,python):2024-01-10-11:37:02.319.746 [mindspore/core/utils/ms_context.cc:225] set_backend_policy] ms set context backend policy:ge [INFO] RUNTIME(164038,python):2024-01-10-11:37:02.319.903 [runtime.cc:1159] 164038 GetAicoreNumByLevel: workingDev_=0 [INFO] RUNTIME(164038,python):2024-01-10-11:37:02.319.942 [runtime.cc:4719] 164038 GetVisibleDevices: ASCEND_RT_VISIBLE_DEVICES param was not set [INFO] DEVICE(164038,ffff87d23440,python):2024-01-10-11:37:02.320.725 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:580] SetContextSocVersion] The soc version :Ascend910A [INFO] PIPELINE(164038,ffff87d23440,python):2024-01-10-11:37:02.364.305 [mindspore/ccsrc/pybind_api/ir/log_adapter_py.h:34] PyExceptionInitializer] Set exception handler [INFO] PIPELINE(164038,ffff87d23440,python):2024-01-10-11:37:02.375.527 [mindspore/ccsrc/pipeline/jit/ps/init.cc:179] pybind11_init__c_expression] Start GraphExecutorPy... [INFO] PIPELINE(164038,ffff87d23440,python):2024-01-10-11:37:02.376.168 [mindspore/ccsrc/pipeline/jit/ps/init.cc:271] pybind11_init__c_expression] Start ParallelContext... [INFO] PIPELINE(164038,ffff87d23440,python):2024-01-10-11:37:02.376.785 [mindspore/ccsrc/pipeline/jit/ps/init.cc:379] pybind11_init__c_expression] Start CostModelContext... [INFO] PIPELINE(164038,ffff87d23440,python):2024-01-10-11:37:02.377.215 [mindspore/ccsrc/pipeline/jit/ps/init.cc:481] pybind11_init__c_expression] Start OffloadContext... [INFO] DEVICE(164038,ffff87d23440,python):2024-01-10-11:37:02.379.342 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:580] SetContextSocVersion] The soc version :Ascend910A [INFO] CORE(164038,ffff87d23440,python):2024-01-10-11:37:04.487.844 [mindspore/core/utils/ms_context.cc:362] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] PARALLEL(164038,ffff87d23440,python):2024-01-10-11:37:04.487.927 [mindspore/ccsrc/frontend/parallel/costmodel_context.cc:30] GetInstance] Create costmodel_context [INFO] ME(164038:281472960443456,MainProcess):2024-01-10-11:37:06.394.299 [mindspore/context.py:382] Reset device target to CPU when set_device_target. [INFO] CORE(164038,ffff87d23440,python):2024-01-10-11:37:06.394.608 [mindspore/core/utils/ms_context.cc:225] set_backend_policy] ms set context backend policy:ms [INFO] CORE(164038,ffff87d23440,python):2024-01-10-11:37:06.394.722 [mindspore/core/utils/ms_context.cc:362] SetDeviceTargetFromInner] ms set context device target:CPU [INFO] ME(164038:281472960443456,MainProcess):2024-01-10-11:37:06.394.830 [mindspore/context.py:382] Reset device target to CPU when set_device_target. [INFO] CORE(164038,ffff87d23440,python):2024-01-10-11:37:06.394.978 [mindspore/core/utils/ms_context.cc:362] SetDeviceTargetFromInner] ms set context device target:CPU [INFO] PS(164038,ffff87d23440,python):2024-01-10-11:37:06.395.189 [mindspore/ccsrc/ps/ps_context.cc:256] set_ms_role] MS_ROLE of this node is MS_SCHED [INFO] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:06.395.280 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:11 [INFO] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:06.395.466 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:188] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281470818099440 [INFO] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:06.395.489 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:13 [INFO] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:06.395.572 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:188] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281470809706736 [INFO] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:06.395.656 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:14 [INFO] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:06.395.672 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:215] StartServerSocket] Start server succ, fd: 14, url: 127.0.0.1:10969 [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:06.395.745 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:195] BuildCluster] Topology build timed out., retry(1/210). [INFO] DISTRIBUTED(164038,ffff072080f0,python):2024-01-10-11:37:06.395.783 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:424] UpdateTopoState] The cluster topology is in the process of constructing, current alive node num: (0/4) [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.490.856 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:15 [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.490.958 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:16 [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.491.092 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:15 [WARNING] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.491.184 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:202] ProcessRegister] The new node: 0(role: MS_WORKER), rank id: 0 is registered successfully. [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.614.782 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:17 [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.614.952 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:18 [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.615.113 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:17 [WARNING] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.615.153 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:202] ProcessRegister] The new node: 2(role: MS_WORKER), rank id: 2 is registered successfully. [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.627.578 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:19 [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.627.748 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:20 [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.627.871 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:19 [WARNING] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.627.904 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:202] ProcessRegister] The new node: 1(role: MS_WORKER), rank id: 1 is registered successfully. [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.882.312 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:21 [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.882.483 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:22 [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.882.616 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:21 [WARNING] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.882.649 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:202] ProcessRegister] The new node: 3(role: MS_WORKER), rank id: 3 is registered successfully. [WARNING] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.882.665 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:604] ReassignNodeRank] Rank ids are already set by numeric node ids, and this is not CM initialization. No need to reassign them. [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.882.720 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:509] AssignPortRange] The port range for node 0, rank id: 0, min port: 8118, max port: 9141 [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.882.732 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:509] AssignPortRange] The port range for node 1, rank id: 1, min port: 9142, max port: 10165 [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.882.744 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:509] AssignPortRange] The port range for node 2, rank id: 2, min port: 10166, max port: 11189 [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.882.758 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:509] AssignPortRange] The port range for node 3, rank id: 3, min port: 11190, max port: 12213 [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:06.882.778 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:477] TransitionToInitialized] The cluster topology has been constructed successfully. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:09.395.875 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:197] BuildCluster] Cluster is successfully initialized. [INFO] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:09.395.955 [mindspore/ccsrc/distributed/init.cc:132] InitializeCollective] Scheduler node does not need to initialize collective communication. [INFO] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:09.395.970 [mindspore/ccsrc/distributed/init.cc:52] Initialize] Scheduler starts to wait for cluster to exit. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:09.395.987 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:09.395.999 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:09.883.128 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:22 [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:12.491.759 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:16 [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:12.615.748 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:18 [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:37:12.628.330 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:20 [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:14.396.133 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:14.396.198 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:19.396.342 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:19.396.411 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:24.396.550 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:24.396.638 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:29.396.775 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:29.396.828 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:34.396.960 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:34.397.015 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:39.397.122 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:39.397.168 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:44.397.269 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:44.397.310 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:49.397.426 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:49.397.477 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:54.397.586 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:54.397.632 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:59.397.816 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:37:59.397.870 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:04.398.001 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:04.398.070 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:09.398.238 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:09.398.312 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:14.398.442 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:14.398.493 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:19.398.617 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:19.398.685 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:24.398.811 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:24.398.871 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:29.398.993 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:29.399.044 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:34.399.170 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:34.399.232 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:39.399.372 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:39.399.445 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:44.399.575 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:44.399.630 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:49.399.762 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:49.399.859 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:54.399.995 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:54.400.062 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:59.400.194 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:38:59.400.250 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:04.400.381 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:04.400.450 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:09.400.588 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:09.400.656 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:14.400.787 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:14.400.841 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:19.400.971 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:19.401.046 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:24.401.180 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:24.401.250 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:29.401.380 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:29.401.434 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:34.401.581 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:34.401.648 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:39.401.809 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:39.401.880 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:44.402.005 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:44.402.055 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:49.402.185 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:49.402.250 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:54.402.392 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:54.402.462 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:59.402.602 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:39:59.402.658 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:04.402.794 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:04.402.849 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:09.402.981 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:09.403.043 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:14.403.191 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:14.403.238 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:19.403.370 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:19.403.428 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:24.403.577 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:24.403.644 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:29.403.776 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:29.403.832 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:34.403.967 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:34.404.027 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:39.404.176 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:39.404.245 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:44.404.381 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:44.404.434 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:49.404.559 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:49.404.615 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:54.404.734 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:54.404.818 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:59.404.935 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:40:59.404.978 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:41:04.405.096 [mindspore/ccsrc/distributed/cluster/topology/meta_server_node.cc:82] Finalize] The meta server node can not be finalized because there are still 4 alive nodes. [WARNING] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:41:04.405.151 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:113] Finalize] This log means the cluster is successfully created. Retry to finalize the node and exit cluster... [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:41:06.514.442 [mindspore/ccsrc/distributed/rpc/tcp/connection.cc:76] SocketEventHandler] Event value fd: 16, events: 8193, state: 4, errcode: 11, errno: 11, to: 127.0.0.1:46012, type:1, remote: 1 [INFO] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:41:09.405.291 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:533] Finalize] Delete send event loop [INFO] DISTRIBUTED(164038,ffff07a090f0,python):2024-01-10-11:41:09.405.416 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:41:09.405.558 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:209] Finalize] Stop loop succ [INFO] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:41:09.405.577 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:540] Finalize] Delete recv event loop [INFO] DISTRIBUTED(164038,ffff0820a0f0,python):2024-01-10-11:41:09.405.605 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:41:09.405.734 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:209] Finalize] Stop loop succ [INFO] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:41:09.405.763 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:554] Finalize] Delete connection pool. [INFO] DISTRIBUTED(164038,ffff87d23440,python):2024-01-10-11:41:12.401.396 [mindspore/ccsrc/distributed/init.cc:54] Initialize] Scheduler ends waiting for cluster to exit. [INFO] RUNTIME(164038,python):2024-01-10-11:41:12.762.382 [runtime.cc:1737] 164038 ~Runtime: deconstruct runtime. [INFO] ATRACE(164039,python):2024-01-10-11:37:02.028.685 [trace_attr.c:105](tid:164039) platform is 1. [INFO] ATRACE(164039,python):2024-01-10-11:37:02.028.906 [trace_recorder.c:114](tid:164039) use root path: /home/jenkins/ascend/atrace [INFO] ATRACE(164039,python):2024-01-10-11:37:02.028.936 [trace_signal.c:133](tid:164039) register signal handler for signo 2 succeed. [INFO] ATRACE(164039,python):2024-01-10-11:37:02.028.947 [trace_signal.c:133](tid:164039) register signal handler for signo 15 succeed. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:02.439.192 [mindspore/core/utils/ms_context.cc:225] set_backend_policy] ms set context backend policy:ge [INFO] RUNTIME(164039,python):2024-01-10-11:37:02.439.379 [runtime.cc:1159] 164039 GetAicoreNumByLevel: workingDev_=0 [INFO] RUNTIME(164039,python):2024-01-10-11:37:02.439.423 [runtime.cc:4719] 164039 GetVisibleDevices: ASCEND_RT_VISIBLE_DEVICES param was not set [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:02.440.167 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:580] SetContextSocVersion] The soc version :Ascend910A [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:02.481.355 [mindspore/ccsrc/pybind_api/ir/log_adapter_py.h:34] PyExceptionInitializer] Set exception handler [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:02.491.576 [mindspore/ccsrc/pipeline/jit/ps/init.cc:179] pybind11_init__c_expression] Start GraphExecutorPy... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:02.492.228 [mindspore/ccsrc/pipeline/jit/ps/init.cc:271] pybind11_init__c_expression] Start ParallelContext... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:02.492.798 [mindspore/ccsrc/pipeline/jit/ps/init.cc:379] pybind11_init__c_expression] Start CostModelContext... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:02.493.192 [mindspore/ccsrc/pipeline/jit/ps/init.cc:481] pybind11_init__c_expression] Start OffloadContext... [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:02.495.221 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:580] SetContextSocVersion] The soc version :Ascend910A [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:04.599.222 [mindspore/core/utils/ms_context.cc:362] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:37:04.599.329 [mindspore/ccsrc/frontend/parallel/costmodel_context.cc:30] GetInstance] Create costmodel_context [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:06.488.469 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:580] SetContextSocVersion] The soc version :Ascend910A [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:06.489.168 [mindspore/core/utils/ms_context.cc:362] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:06.489.299 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:580] SetContextSocVersion] The soc version :Ascend910A [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:06.489.813 [mindspore/core/utils/ms_context.cc:362] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] PS(164039,ffff87fd6440,python):2024-01-10-11:37:06.490.049 [mindspore/ccsrc/ps/ps_context.cc:256] set_ms_role] MS_ROLE of this node is MS_WORKER [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:06.490.111 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:11 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:06.490.268 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:188] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281470820765936 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:06.490.291 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:13 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:06.490.377 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:188] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281470812373232 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:06.490.415 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:15 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:06.490.491 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:188] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281470803980528 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:06.490.509 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:17 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:06.490.588 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:188] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281470795587824 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:06.490.626 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:412] Connect] Can not found link destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:06.490.794 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:18 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:06.490.828 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:456] Connect] Connection 18 source: 127.0.0.1:46010, destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:06.490.848 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:475] Connect] Connected to destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:06.490.860 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:67] Connect] Connected to the tcp server 127.0.0.1:10969 successfully. [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:06.490.874 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:412] Connect] Can not found link destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164039,ffff074930f0,python):2024-01-10-11:37:06.490.878 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:18 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:06.490.939 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:19 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:06.490.963 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:456] Connect] Connection 19 source: 127.0.0.1:46012, destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:06.490.979 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:475] Connect] Connected to destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:06.490.990 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:67] Connect] Connected to the tcp server 127.0.0.1:10969 successfully. [INFO] DISTRIBUTED(164039,ffff084950f0,python):2024-01-10-11:37:06.490.998 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:19 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:06.491.330 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:209] Register] The compute graph node: 0 has been registered successfully. [WARNING] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:06.491.400 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:195] BuildCluster] Topology build timed out., retry(1/210). [INFO] DISTRIBUTED(164039,ffff064910f0,python):2024-01-10-11:37:06.491.456 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:247] Heartbeat] The heartbeat thread is started. [WARNING] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:09.491.478 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:195] BuildCluster] Topology build timed out., retry(2/210). [WARNING] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:12.491.575 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:197] BuildCluster] Cluster is successfully initialized. [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:12.491.611 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:255] PostProcess] Start post processing for computing graph nodes. [WARNING] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:12.491.939 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:261] PostProcess] This node 0 rank id: 0 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:12.491.959 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:268] PostProcess] Client ip address in this cluster of this compute graph node is 127.0.0.1 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:12.492.054 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:279] PostProcess] Port range assigned for this node 0 is 8118 to 9141 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:12.492.084 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:133] node_num] Number of role MS_WORKER is 4 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:12.492.110 [mindspore/ccsrc/distributed/collective/collective_manager.cc:157] Initialize] Start initializing collective communication for backend: Ascend... [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:12.492.178 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:37] MsCollectiveCommLib] Global group name of MindSpore collective communication library is mccl_world_group [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:12.492.191 [mindspore/ccsrc/distributed/collective/collective_manager.cc:412] InitHostCommlib] Start initializing communication library on host side... [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:12.492.226 [mindspore/ccsrc/distributed/collective/collective_manager.cc:432] InitHostCommlib] Communication library on host side is successfully initialized. Global rank id: 0, global rank size: 4 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:12.492.240 [mindspore/ccsrc/distributed/collective/collective_manager.cc:470] AssignLocalRank] Host name for rank 0 is ascend85 [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:12.492.388 [mindspore/ccsrc/distributed/collective/collective_manager.cc:505] AssignLocalRank] The local rank id assigned for this process is 0. device_id of ms_context is set. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:12.503.161 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:624] SetRtDevice] Enter SetRtDevice, current initialize device number:0 [INFO] TDT(164039,python):2024-01-10-11:37:12.503.881 [process_mode_manager.cpp:109][OpenProcess][tid:164039] [ProcessModeManager] enter into open process deviceId[0] rankSize[0] [INFO] TDT(164039,python):2024-01-10-11:37:12.505.949 [process_mode_manager.cpp:379][InitTsdClient][tid:164039] [TsdClient] deviceId[0] begin to init hdc client [INFO] TDT(164039,python):2024-01-10-11:37:12.506.102 [version_verify.cpp:34][SetVersionInfo][tid:164039] VersionVerify: send client version to server [INFO] TDT(164039,python):2024-01-10-11:37:12.506.147 [version_verify.cpp:50][SetVersionInfo][tid:164039] send feature_info:{msg_type:35, features:{check before send aicpu package,}} [INFO] TDT(164039,python):2024-01-10-11:37:12.506.160 [version_verify.cpp:50][SetVersionInfo][tid:164039] send feature_info:{msg_type:37, features:{check before send open qs message,}} [INFO] TDT(164039,python):2024-01-10-11:37:12.506.435 [version_verify.cpp:66][PeerVersionCheck][tid:164039] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(164039,python):2024-01-10-11:37:12.506.450 [version_verify.cpp:87][ParseVersionInfo][tid:164039] VersionVerify: pass client version info success [INFO] TDT(164039,python):2024-01-10-11:37:12.506.460 [hdc_client.cpp:276][CheckHdcConnection][tid:164039] Service[2] create hdc success [INFO] TDT(164039,python):2024-01-10-11:37:12.506.475 [version_verify.cpp:120][SpecialFeatureCheck][tid:164039] VersionVerify: new type[35], supported [INFO] TDT(164039,python):2024-01-10-11:37:12.506.515 [process_mode_manager.cpp:748][GetDeviceCheckCode][tid:164039] [TsdClient][deviceId=0] [sessionId=1] wait package info respond [INFO] TDT(164039,python):2024-01-10-11:37:12.506.636 [process_mode_manager.cpp:379][InitTsdClient][tid:164039] [TsdClient] deviceId[0] begin to init hdc client [INFO] TDT(164039,python):2024-01-10-11:37:12.506.768 [version_verify.cpp:34][SetVersionInfo][tid:164039] VersionVerify: send client version to server [INFO] TDT(164039,python):2024-01-10-11:37:12.506.780 [version_verify.cpp:50][SetVersionInfo][tid:164039] send feature_info:{msg_type:35, features:{check before send aicpu package,}} [INFO] TDT(164039,python):2024-01-10-11:37:12.506.790 [version_verify.cpp:50][SetVersionInfo][tid:164039] send feature_info:{msg_type:37, features:{check before send open qs message,}} [INFO] TDT(164039,python):2024-01-10-11:37:12.506.927 [version_verify.cpp:66][PeerVersionCheck][tid:164039] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(164039,python):2024-01-10-11:37:12.506.938 [version_verify.cpp:87][ParseVersionInfo][tid:164039] VersionVerify: pass client version info success [INFO] TDT(164039,python):2024-01-10-11:37:12.506.947 [hdc_client.cpp:276][CheckHdcConnection][tid:164039] Service[2] create hdc success [INFO] TDT(164039,python):2024-01-10-11:37:12.506.958 [process_mode_manager.cpp:426][ConstructOpenMsg][tid:164039] [TsdClient] tsd get process sign successfully, procpid[164039] signSize[48] [INFO] TDT(164039,python):2024-01-10-11:37:12.506.969 [version_verify.cpp:112][SpecialFeatureCheck][tid:164039] VersionVerify: previous type[6], supported [INFO] TDT(164039,python):2024-01-10-11:37:12.506.989 [process_mode_manager.cpp:126][OpenProcess][tid:164039] [ProcessModeManager] deviceId[0] sessionId[1] rankSize[0], wait sub process start respond [INFO] TDT(164039,python):2024-01-10-11:37:12.712.386 [stub_process_mode_nowin.cpp:63][ProcessQueueForMdc][tid:164039] [TsdClient] it is unnecessary of current mode[0] chiptype[1] to grant queue auth to aicpusd [INFO] TDT(164039,python):2024-01-10-11:37:12.712.413 [stub_process_mode_nowin.cpp:101][OpenInHost][tid:164039] enter into OpenInHost deviceid[0] [INFO] TDT(164039,python):2024-01-10-11:37:12.712.423 [stub_process_mode_nowin.cpp:105][OpenInHost][tid:164039] host cpu not support [INFO] TDT(164039,python):2024-01-10-11:37:12.712.431 [process_mode_manager.cpp:156][OpenProcess][tid:164039] [TsdClient][deviceId=0] [sessionId=1] start hccp and computer process success [INFO] RUNTIME(164039,python):2024-01-10-11:37:12.715.126 [device.cc:340] 164039 Init: isDoubledie:0, topologytype:0 [INFO] RUNTIME(164039,python):2024-01-10-11:37:12.729.158 [npu_driver.cc:5428] 164519 GetDeviceStatus: GetDeviceStatus status=1. [INFO] ATRACE(164039,python):2024-01-10-11:37:12.729.239 [atrace_api.c:28](tid:164039) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:12.729.380 [trace_rb_log.c:84](tid:164039) [RUNTIME_ATRACE_DEV0_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:12.729.397 [atrace_api.c:32](tid:164039) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:12.729.422 [client_manager.cpp:157][SetProfilingCallback][tid:164039] [TsdClient] set profiling callback success [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:12.734.833 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:646] CreateDefaultStream] Create ascend default stream, stream id: 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:12.736.040 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:652] CreateDefaultStream] Create ascend communication stream, stream id: 1 [INFO] DEBUG(164039,ffff87fd6440,python):2024-01-10-11:37:12.736.119 [mindspore/ccsrc/debug/debugger/debugger.cc:101] Debugger] Debugger got device_target: Ascend [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:12.736.315 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:116] Initialize] Device HBM Size:32768M, Device free HBM Size:32737M, Reserved HBM size for Other Components(HCCL/rts/etc.):2057M, Recommend Reserved HBM size for Other Components:2046M, User define MindSpore HBM Size:0G, MindSpore Used HBM Size:30680M. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:12.805.194 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:301] MallocFromRts] Call rtMalloc to allocate device memory Success, size: 32170311680 bytes, address start: 0x124100000000 end: 0x12487d800000 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:12.805.230 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:129] Initialize] Ascend Memory Adapter initialize success, Memory Statistics: Device HBM memory size: 32768M MindSpore Used memory size: 30680M MindSpore memory base address: 0x124100000000 Total Static Memory size: 0M Total Dynamic memory size: 0M Dynamic memory size of this graph: 0M [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:12.806.477 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:463] SetDisableReuseMemoryFlag] DISABLE_REUSE_MEMORY is not set in ENV. Now set to default value 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:12.806.507 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:500] SetHcclOptions] No hccl mode. If use hccl, make sure [RANK_TABLE_FILE,RANK_ID,DEVICE_ID] all be set in ENV. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:12.806.561 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:366] GetGeOptions] JOB_ID is not set in ENV. Now set to default value 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:12.806.578 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:384] GetGeOptions] Set proto lib path failed! [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:12.806.590 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:304] SetAscendConfig] GE topo sorting mode is: [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:12.806.603 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:316] SetAscendConfig] Set GE topo mode to memory-priority. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:12.806.615 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:321] SetAscendConfig] Set staticMemoryPolicy to default mode. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:12.806.626 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:329] SetAscendConfig] The default value of jit_compile is set to 2. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:12.806.637 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:285] SetAscendHF32Config] The default value of allow_matmul_hf32 and allow_conv_hf32 are set by CANN. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:12.806.647 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:294] SetAscendHF32Config] allow_matmul_hf32: , allow_conv_hf32: [TRACE] GE(164039,python):2024-01-10-11:37:12.806.700 [status:INIT] [ge_api.cc:144]164039 GEInitializeImpl:GEInitialize start [INFO] PROFILING(164039,python):2024-01-10-11:37:13.022.344 [msprofiler_impl.cpp:156] >>> (tid:164039) ProfNotifySetDevice called, is open: 1, devId: 0 [INFO] PROFILING(164039,python):2024-01-10-11:37:13.022.482 [platform.cpp:38] >>> (tid:164039) Profiling platform version: 1.0. [INFO] PROFILING(164039,python):2024-01-10-11:37:13.022.497 [ai_drv_dev_api.cpp:384] >>> (tid:164039) Succeeded to DrvGetApiVersion version: 0x72313 [TRACE] GE(164039,python):2024-01-10-11:37:13.069.970 [status:RUNNING] [ge_api.cc:211]164039 GEInitializeImpl:Initializing environment [INFO] GE(164039,python):2024-01-10-11:37:13.070.027 [gelib.cc:98][EVENT]164039 Initialize:[GEPERFTRACE] GE Init Start [INFO] GE(164039,python):2024-01-10-11:37:13.070.340 [gelib.cc:307][EVENT]164039 SystemInitialize:Online infer init GELib success, device id :0 [INFO] DVPP(164039,python):2024-01-10-11:37:13.400.522 [dvpp_engine.cc:41][ENGINE][Initialize:41][tid:164039]dvpp engine do not support [INFO] TUNE(164039,python):2024-01-10-11:37:13.406.094 [cann_kb_pyfunc_mgr.cpp:72][CANNKB][Tid:164039]"CannKbPyfuncMgr: Enter PyObjectInit, reference_ is 0!" [INFO] TUNE(164039,python):2024-01-10-11:37:13.406.131 [handle_manager.cpp:115][CANNKB][Tid:164039]"Start to run init functions to load dynamic python lib!" [INFO] TUNE(164039,python):2024-01-10-11:37:13.406.185 [handle_manager.cpp:407][CANNKB][Tid:164039]"Init functions of loading dynamic python lib end!" [INFO] TUNE(164039,python):2024-01-10-11:37:13.406.196 [cann_kb_pyfunc_mgr.cpp:24][CANNKB][Tid:164039]"CANN_KB_Py has already been initialized." [INFO] TUNE(164039,python):2024-01-10-11:37:13.406.255 [cann_kb_pyfunc_mgr.cpp:117][CANNKB][Tid:164039]"CannKbPyfuncMgr: Run PyObjectInit successfully!" [INFO] HCCL(164039,python):2024-01-10-11:37:24.914.614 [plugin_manager.cc:42][164039]hcom running normal mode. [INFO] DVPP(164039,python):2024-01-10-11:37:24.915.274 [dvpp_engine.cc:92][ENGINE][GetOpsKernelInfoStores:92][tid:164039]dvpp ops kernel info store do not support [INFO] DVPP(164039,python):2024-01-10-11:37:24.915.397 [dvpp_engine.cc:69][ENGINE][GetGraphOptimizerObjs:69][tid:164039]dvpp graph optimizer do not support [INFO] DVPP(164039,python):2024-01-10-11:37:25.568.274 [dvpp_ops_kernel_builder.cc:48][ENGINE][Initialize:48][tid:164039]dvpp ops kernel builder do not support [INFO] GE(164039,python):2024-01-10-11:37:25.577.036 [gelib.cc:169][EVENT]164039 Initialize:[GEPERFTRACE] The time cost of GELib::Initialize is [12506920] micro second. [TRACE] GE(164039,python):2024-01-10-11:37:25.659.185 [status:STOP] [ge_api.cc:255]164039 GEInitializeImpl:GEInitialize finished [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:25.659.381 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_res_manager.cc:168] GeSetContextOptions] Set GE atomic clean policy to 1. [TRACE] GE(164039,python):2024-01-10-11:37:25.659.452 [status:INIT] [ge_api.cc:398]164039 Session:Start to construct session. [TRACE] GE(164039,python):2024-01-10-11:37:25.659.470 [status:RUNNING] [ge_api.cc:408]164039 Session:Creating session [INFO] GE(164039,python):2024-01-10-11:37:25.659.910 [graph_var_manager.cc:1445][EVENT]164039 SetMemoryMallocSize:Total memory size is 34359738368 [INFO] GE(164039,python):2024-01-10-11:37:25.659.927 [graph_var_manager.cc:1424][EVENT]164039 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] PROFILING(164039,python):2024-01-10-11:37:25.660.275 [msprofiler_impl.cpp:156] >>> (tid:164039) ProfNotifySetDevice called, is open: 1, devId: 0 [TRACE] GE(164039,python):2024-01-10-11:37:25.661.072 [status:RUNNING] [ge_api.cc:411]164039 Session:Session id is 0 [TRACE] GE(164039,python):2024-01-10-11:37:25.661.093 [status:STOP] [ge_api.cc:420]164039 Session:Session Constructor finished [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:25.661.124 [mindspore/ccsrc/transform/graph_ir/graph_runner.cc:53] NewSession] Create new GE session success! [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:25.661.148 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:210] SetGeSession] Add a new Ge Session success [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:25.661.192 [mindspore/ccsrc/transform/graph_ir/graph_runner.cc:65] GraphRunner] ME run in ONE_DEVICE strategy mode [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:25.661.240 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:246] SetGraphRunner] Add a new GraphRunner success [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:25.661.286 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:238] InitGe] Create session and graphrunner successful. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:25.661.303 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:242] InitGe] Init ge successful, ge reference = 1. [INFO] PROFILING(164039,python):2024-01-10-11:37:25.670.860 [platform.cpp:38] >>> (tid:164039) Profiling platform version: 1.0. [INFO] PROFILING(164039,python):2024-01-10-11:37:25.670.885 [ai_drv_dev_api.cpp:384] >>> (tid:164039) Succeeded to DrvGetApiVersion version: 0x72313 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:25.671.042 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:193] Initialize] Call aclInit successfully. [TRACE] GE(164039,python):2024-01-10-11:37:25.671.152 [status:INIT] [ge_api.cc:144]164039 GEInitializeImpl:GEInitialize start [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:25.672.430 [mindspore/ccsrc/distributed/collective/collective_manager.cc:455] InitDeviceCommLib] Start initializing communication library on device side... [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:25.672.503 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:291] OpenTsd] Device id = 0, rank size = 4. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:25.681.009 [mindspore/ccsrc/plugin/device/ascend/hal/device/tensorprint_utils.cc:405] CreateChannel] For Print ops, select MBUF channel. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:25.681.044 [mindspore/ccsrc/plugin/device/ascend/hal/device/tensorprint_utils.cc:420] CreateTensorPrintThread] Success to create acl channel handle, tsd reference = 1. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:25.681.789 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:103] MbufDataHandler] Channel ms_tensor_dump begins the construction process. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:25.682.319 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:103] MbufDataHandler] Channel ms_tensor_summary begins the construction process. [INFO] DEVICE(164039,fffebef100f0,python):2024-01-10-11:37:25.682.358 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:161] HandleData] Channel ms_tensor_dump starts executing HandleData. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:25.682.817 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:103] MbufDataHandler] Channel ms_image_summary begins the construction process. [INFO] DEVICE(164039,fffebe70f0f0,python):2024-01-10-11:37:25.682.866 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:161] HandleData] Channel ms_tensor_summary starts executing HandleData. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:25.683.337 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:103] MbufDataHandler] Channel ms_scalar_summary begins the construction process. [INFO] DEVICE(164039,fffebdf0e0f0,python):2024-01-10-11:37:25.683.408 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:161] HandleData] Channel ms_image_summary starts executing HandleData. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:25.683.657 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:103] MbufDataHandler] Channel ms_histogram_summary begins the construction process. [INFO] DEVICE(164039,fffebd70d0f0,python):2024-01-10-11:37:25.683.710 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:161] HandleData] Channel ms_scalar_summary starts executing HandleData. [INFO] HCCL_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:25.683.974 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:202] InitHccl] Start init hccl adapter. [INFO] DEVICE(164039,fffebcf0c0f0,python):2024-01-10-11:37:25.684.004 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:161] HandleData] Channel ms_histogram_summary starts executing HandleData. [INFO] HCCL_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:25.684.121 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:341] InitKernelInfoStore] Start init hccl kernel info store. [INFO] HCCL_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:25.684.195 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:358] InitKernelInfoStore] Get builder ops_kernel_info_hccl [INFO] HCCL(164039,python):2024-01-10-11:37:25.684.222 [plugin_manager.cc:42][164039]hcom running normal mode. [INFO] HCCL_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:25.684.292 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:380] InitKernelInfoStore] Init hccl kernel info store success. [INFO] HCCL_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:25.684.311 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:528] InitHcclExec] Start init hccl exec. [INFO] HCCL(164039,python):2024-01-10-11:37:25.687.136 [hcom_executor.cc:32][164039][Initialize][HcomExecutor]Hcom Excutor Initialize end. ret[0] [INFO] HCCL_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:25.687.179 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:534] InitHcclExec] Hcom DynamicKernel Initialize success [INFO] HCCL_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:25.687.199 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:540] InitHcclExec] InitHcclExec success [INFO] HCCL_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:25.687.211 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:220] InitHccl] Init hccl adapter success. [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:25.687.229 [mindspore/ccsrc/distributed/collective/collective_manager.cc:458] InitDeviceCommLib] Communication library on device side is successfully initialized. [WARNING] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:25.687.246 [mindspore/ccsrc/distributed/collective/collective_manager.cc:220] CreateCommunicationGroup] Start to create communication group: hccl_world_group [const vector]{0, 1, 2, 3} [INFO] HCCL(164039,python):2024-01-10-11:37:25.687.302 [op_base.cc:308][164039]Entry-HcclGetRootInfo:rootInfo[0x49607df0], deviceLogicId[0] [INFO] HCCL(164039,python):2024-01-10-11:37:25.687.329 [externalinput.cc:310][164039]environmental variable HCCL_CONNECT_TIMEOUT is set, timeOut[600] [INFO] HCCL(164039,python):2024-01-10-11:37:25.687.350 [externalinput.cc:598][164039]environmental variable HCCL_IF_IP is not set [INFO] HCCL(164039,python):2024-01-10-11:37:25.687.362 [externalinput.cc:655][164039]environmental variable HCCL_SOCKET_IFNAME is not set, default[EmptyString] [INFO] HCCL(164039,python):2024-01-10-11:37:25.687.377 [externalinput.cc:582][164039]environmental variable HCCL_IF_BASE_PORT is not set [INFO] HCCL(164039,python):2024-01-10-11:37:25.687.464 [externalinput.cc:282][164039]environmental variable HCCL_HIGH_PERF_ENABLE is not set [INFO] HCCP(164039,python):2024-01-10-11:37:25.687.753 [ra_host.c:1882]tid:164039,ra_get_ifnum(1882) : Input parameters: phy_id[0], nic_position:[0] [INFO] HCCL(164039,python):2024-01-10-11:37:25.689.754 [adapter_hccp.cc:821][164039][Get][HostIf]hrtGetIfNum success. ifAddrNum[7]. [INFO] HCCP(164039,python):2024-01-10-11:37:25.689.774 [ra_host.c:1930]tid:164039,ra_get_ifaddrs(1930) : Input parameters: phy_id[0], nic_position:[0], interface num[7] [INFO] HCCL(164039,python):2024-01-10-11:37:25.690.940 [sal.cc:383][164039]nic class[normal]: find nic[8.92.9.85%enp189s0f0] success. [INFO] HCCP(164039,python):2024-01-10-11:37:25.691.010 [ra_host.c:1722]tid:164039,ra_socket_set_white_list_status(1722) : Input parameters: enable[0] [INFO] HCCP(164039,python):2024-01-10-11:37:25.691.029 [ra_host.c:293]tid:164039,ra_init(293) : Input parameters: phy_id[0], nic_position:[0] [INFO] HCCP(164039,python):2024-01-10-11:37:25.691.312 [rs_ssl.c:1104]tid:164039,rs_ssl_init(1104) : TLS SWITCH (0) [INFO] HCCP(164039,python):2024-01-10-11:37:25.691.448 [rs_epoll.c:470]tid:166201,rs_epoll_handle(470) : pthread[epoll_pthread] is alive! [INFO] HCCP(164039,python):2024-01-10-11:37:25.691.486 [rs.c:403]tid:164039,rs_init(403) : rs init success, chip_id[0] [INFO] HCCP(164039,python):2024-01-10-11:37:25.691.479 [rs_epoll.c:595]tid:166202,rs_connect_handle(595) : pthread[connect_pthread] is alive! [INFO] HCCP(164039,python):2024-01-10-11:37:25.691.520 [ra_host.c:454]tid:164039,ra_socket_init_v1(454) : socket init:mode=0 phy_id=0 family=2 ip=8.92.9.85 [INFO] HCCP(164039,python):2024-01-10-11:37:25.691.546 [ra_host.c:903]tid:164039,ra_socket_listen_start(903) : Input parameters: [0]th, phy_id[0], local_ip[8.92.9.85] [INFO] HCCP(164039,python):2024-01-10-11:37:25.691.585 [rs_socket.c:646]tid:164039,rs_socket_listen_bind_listen(646) : socket bind: family 2, addr 8.92.9.85, port 60000 [TRACE] HCCL(164039,python):2024-01-10-11:37:25.691.671 [status:init] [op_base.cc:331][164039]HcclGetRootInfo success, take time [4376]us [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:25.691.840 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:175] SendUniqueID] The unique id for group hccl_world_group has been registered to the meta server. [WARNING] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:25.691.865 [mindspore/ccsrc/distributed/collective/collective_manager.cc:278] CreateCommunicationGroup] Begin initialize communication group on the device side: hccl_world_group [INFO] HCCL(164039,python):2024-01-10-11:37:25.691.984 [op_base.cc:405][166204]Entry-HcclCommInitRootInfo:ranks[4], rank[0], rootinfo: host ip[8.92.9.85] port[60000] nicDeploy[1] identifier[8.92.9.85%enp189s0f0_60000_0_1704857845691656], deviceLogicId[0] [INFO] TDT(164039,python):2024-01-10-11:37:25.692.159 [process_mode_manager.cpp:109][OpenProcess][tid:166204] [ProcessModeManager] enter into open process deviceId[0] rankSize[2] [INFO] TDT(164039,python):2024-01-10-11:37:25.692.500 [process_mode_manager.cpp:705][GetDeviceCheckCode][tid:166204] [ProcessModeManager][deviceId=0] aicpu package already exist in device [INFO] TDT(164039,python):2024-01-10-11:37:25.692.536 [process_mode_manager.cpp:426][ConstructOpenMsg][tid:166204] [TsdClient] tsd get process sign successfully, procpid[164039] signSize[48] [INFO] TDT(164039,python):2024-01-10-11:37:25.692.589 [process_mode_manager.cpp:126][OpenProcess][tid:166204] [ProcessModeManager] deviceId[0] sessionId[1] rankSize[2], wait sub process start respond [INFO] TDT(164039,python):2024-01-10-11:37:25.852.751 [stub_process_mode_nowin.cpp:63][ProcessQueueForMdc][tid:166204] [TsdClient] it is unnecessary of current mode[0] chiptype[1] to grant queue auth to aicpusd [INFO] TDT(164039,python):2024-01-10-11:37:25.852.770 [stub_process_mode_nowin.cpp:101][OpenInHost][tid:166204] enter into OpenInHost deviceid[0] [INFO] TDT(164039,python):2024-01-10-11:37:25.852.780 [stub_process_mode_nowin.cpp:105][OpenInHost][tid:166204] host cpu not support [INFO] TDT(164039,python):2024-01-10-11:37:25.852.789 [process_mode_manager.cpp:156][OpenProcess][tid:166204] [TsdClient][deviceId=0] [sessionId=1] start hccp and computer process success [INFO] HCCP(164039,python):2024-01-10-11:37:25.852.801 [ra_host.c:293]tid:166204,ra_init(293) : Input parameters: phy_id[0], nic_position:[1] [INFO] HCCP(164039,python):2024-01-10-11:37:25.852.834 [ra_hdc.c:1465]tid:166204,ra_hdc_init(1465) : hdc init start! logic id is 0, phy id is 0 [INFO] HCCP(164039,python):2024-01-10-11:37:25.853.190 [ra_hdc.c:1500]tid:166204,ra_hdc_init(1500) : hdc init OK! phy_id[0] [INFO] HCCL(164039,python):2024-01-10-11:37:25.855.130 [adapter_hccp.cc:988][166204][Get][DeviceIP]hrtGetIfNum success. ifAddrNum[2]. [INFO] HCCP(164039,python):2024-01-10-11:37:25.855.142 [ra_host.c:1930]tid:166204,ra_get_ifaddrs(1930) : Input parameters: phy_id[0], nic_position:[1], interface num[2] [INFO] HCCL(164039,python):2024-01-10-11:37:25.859.719 [adapter_hccp.cc:1018][166204]hrtGetIfAddress: idx[0] ifname[eth0] ip[192.168.100.101%eth0] [INFO] HCCL(164039,python):2024-01-10-11:37:25.859.732 [topoinfo_detect.cc:472][166204]select AF_INET family as device socket family. [INFO] HCCP(164039,python):2024-01-10-11:37:25.859.815 [ra_host.c:825]tid:166204,ra_socket_batch_connect(825) : Input parameters: [0]th, phy_id[0], local_ip[8.92.9.85], remote_ip[8.92.9.85], tag:[topo_detect_default_tag_60000] [INFO] HCCP(164039,python):2024-01-10-11:37:26.693.178 [ra_host.c:863]tid:166203,ra_socket_batch_close(863) : Input parameters: [0]th, phy_id[0], local_ip[8.92.9.85] [INFO] HCCP(164039,python):2024-01-10-11:37:26.737.744 [ra_host.c:863]tid:166203,ra_socket_batch_close(863) : Input parameters: [0]th, phy_id[0], local_ip[8.92.9.85] [INFO] HCCP(164039,python):2024-01-10-11:37:26.737.812 [ra_host.c:863]tid:166203,ra_socket_batch_close(863) : Input parameters: [0]th, phy_id[0], local_ip[8.92.9.85] [INFO] HCCP(164039,python):2024-01-10-11:37:26.737.858 [ra_host.c:863]tid:166203,ra_socket_batch_close(863) : Input parameters: [0]th, phy_id[0], local_ip[8.92.9.85] [INFO] HCCP(164039,python):2024-01-10-11:37:26.737.872 [ra_host.c:863]tid:166204,ra_socket_batch_close(863) : Input parameters: [0]th, phy_id[0], local_ip[8.92.9.85] [INFO] HCCP(164039,python):2024-01-10-11:37:26.737.911 [ra_host.c:941]tid:166203,ra_socket_listen_stop(941) : Input parameters: [0]th, phy_id[0], local_ip[8.92.9.85] [INFO] HCCP(164039,python):2024-01-10-11:37:26.738.035 [ra_host.c:525]tid:166204,ra_socket_deinit(525) : Input parameters: phy_id[0] family[2] local_ip[8.92.9.85] [INFO] HCCP(164039,python):2024-01-10-11:37:26.738.061 [rs.c:1257]tid:166204,rs_socket_deinit(1257) : socket deinit success, phy_id:0, local_ip:8.92.9.85 [INFO] HCCP(164039,python):2024-01-10-11:37:26.738.078 [ra_host.c:349]tid:166204,ra_deinit(349) : Input parameters: phy_id[0], nic_position:[0] [INFO] HCCP(164039,python):2024-01-10-11:37:26.958.716 [rs.c:1460]tid:166204,rs_deinit(1460) : rs_deinit chip_id[0] ok [INFO] HCCP(164039,python):2024-01-10-11:37:26.958.737 [ra_host.c:349]tid:166204,ra_deinit(349) : Input parameters: phy_id[0], nic_position:[1] [INFO] HCCP(164039,python):2024-01-10-11:37:26.958.747 [ra_hdc.c:1535]tid:166204,ra_hdc_deinit(1535) : hdc deinit start! phy_id[0] [INFO] HCCP(164039,python):2024-01-10-11:37:26.958.877 [ra_hdc.c:1570]tid:166204,ra_hdc_deinit(1570) : hdc deinit OK! phy_id[0] [INFO] ATRACE(164039,python):2024-01-10-11:37:26.959.117 [atrace_api.c:28](tid:166204) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:26.959.177 [trace_rb_log.c:84](tid:166204) [HCCL_166204_0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:26.959.190 [atrace_api.c:32](tid:166204) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:26.959.641 [process_mode_manager.cpp:109][OpenProcess][tid:166204] [ProcessModeManager] enter into open process deviceId[0] rankSize[2] [INFO] HCCP(164039,python):2024-01-10-11:37:26.959.660 [ra_host.c:293]tid:166204,ra_init(293) : Input parameters: phy_id[0], nic_position:[1] [INFO] HCCP(164039,python):2024-01-10-11:37:26.959.742 [ra_hdc.c:1465]tid:166204,ra_hdc_init(1465) : hdc init start! logic id is 0, phy id is 0 [INFO] HCCP(164039,python):2024-01-10-11:37:26.959.996 [ra_hdc.c:1500]tid:166204,ra_hdc_init(1500) : hdc init OK! phy_id[0] [INFO] HCCP(164039,python):2024-01-10-11:37:26.961.946 [ra_host.c:389]tid:166204,ra_socket_init(389) : socket init:mode=1 phy_id=0 family=2 ip=0.0.0.0 [INFO] HCCP(164039,python):2024-01-10-11:37:26.962.507 [ra_host.c:903]tid:166204,ra_socket_listen_start(903) : Input parameters: [0]th, phy_id[0], local_ip[0.0.0.0] [INFO] HCCL(164039,python):2024-01-10-11:37:26.966.652 [hccl_impl.cc:430][166204]hccl algorithm: [Module(aiserver)] there are 4 device in level0, using fullmesh algo [TRACE] HCCL(164039,python):2024-01-10-11:37:26.968.953 [status:init] [op_base.cc:481][166204]HcclCommInitRootInfo success,take time [1276974]us, rankNum[4], rank[0],rootInfo identifier[8.92.9.85%enp189s0f0_60000_0_1704857845691656], server[8.92.9.85%enp189s0f0], device[0] [WARNING] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:26.969.016 [mindspore/ccsrc/distributed/collective/collective_manager.cc:287] CreateCommunicationGroup] End initialize communication group on the device side: hccl_world_group [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:37:26.969.046 [mindspore/ccsrc/distributed/collective/collective_manager.cc:182] Initialize] End initializing collective communication for backend: Ascend [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:26.969.086 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:285] RecordInitStatus] Status record: system init. [WARNING] ME(164039:281472963273792,MainProcess):2024-01-10-11:37:28.714.988 [mindspore/parallel/_utils.py:259] You are suggested to use mindspore.context.set_auto_parallel_context(parameter_broadcast=True) or mindspore.common.set_seed() to share parameters among multi-devices. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:29.481.251 [mindspore/ccsrc/minddata/dataset/util/task_manager.cc:161] DoServiceStart] Starting Task Manager. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:29.481.898 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at train-labels-idx1-ubyte. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:29.481.931 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at train-images-idx3-ubyte. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:29.482.048 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:29.483.792 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:29.483.895 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:29.483.915 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:29.483.934 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:29.483.969 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:29.484.035 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:29.484.062 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:29.484.078 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:29.484.099 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:29.484.117 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:29.484.149 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:29.484.179 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:29.484.201 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:29.484.230 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:29.484.546 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at train-labels-idx1-ubyte. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:29.484.569 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at train-images-idx3-ubyte. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.108.982 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.109.097 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.153.871 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.154.010 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.154.031 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.154.053 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.154.086 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.154.171 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.154.198 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.154.214 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.154.235 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.154.254 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.154.285 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.154.299 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.154.320 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.154.349 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.154.691 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at train-labels-idx1-ubyte. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.154.717 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at train-images-idx3-ubyte. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.719.408 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.719.526 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.764.129 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.764.286 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.764.306 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.764.325 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.764.358 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.764.428 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.764.456 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.764.471 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.764.491 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.764.509 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.764.541 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.764.555 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.764.576 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.764.607 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.764.932 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] ME(164039:281472963273792,MainProcess):2024-01-10-11:37:30.835.584 [mindspore/dataset/engine/datasets.py:4269] queue_name is newly generated. value is 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.838.372 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.838.504 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.838.527 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.838.548 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.838.586 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.838.657 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.838.686 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.838.703 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.838.724 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.838.743 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.838.775 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.838.790 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.838.811 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.838.840 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:30.839.190 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:30.839.391 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1729] InitExecDatasetVm] Start InitDataSet Entry [INFO] DEBUG(164039,ffff87fd6440,python):2024-01-10-11:37:30.839.722 [mindspore/ccsrc/common/debug/env_config_parser.cc:152] ParseFromFile] The 'env_config_path' in 'mindspore.context.set_context(env_config_path={path})' is empty. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:37:30.839.755 [mindspore/ccsrc/backend/graph_compiler/transform.cc:573] CreateBackend] CreateBackend is: ge [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:30.839.774 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:30.839.787 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEBUG(164039,ffff87fd6440,python):2024-01-10-11:37:30.839.966 [mindspore/ccsrc/debug/debugger/debugger.cc:129] Init] Debugger got device_id: 0 [INFO] DEBUG(164039,ffff87fd6440,python):2024-01-10-11:37:30.839.985 [mindspore/ccsrc/debug/debugger/debugger.cc:131] Init] Debugger got device_target: Ascend [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:30.949.170 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:433] Initialize] The actor thread number: 5, the kernel thread number: 25 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:30.949.504 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:30.949.527 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:30.949.541 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1252] SetRunMode] Run graph mode with kernel by kernel by configuration. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:37:30.950.053 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:546] CompileGraphs] Status record: start compile function graph: _anonymous__1 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.950.184 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unify_mindir_pm_0_erase_invalid_micro_depend in 9.73 us [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:37:30.950.868 [mindspore/ccsrc/utils/anfalgo.cc:1736] IsNodeOutputDynamicShape] Invalid base shape, node: Default/Return-op0 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:30.950.954 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:30.950.970 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:30.951.012 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:30.951.025 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:30.951.047 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:30.951.074 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:37:30.951.105 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:727] CompileGraph] Compile graph: _anonymous__1, Split segments size: 2 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:37:30.951.167 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:762] CompileGraphFromSegment] Compile normal segment, the first node: @_anonymous__1:CNode_2{[0]: ValueNode InitDataSetQueue} [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:30.951.257 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:421] CompileGraph] Status record: start compile graph. [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:30.951.334 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:1940] ConstructKernelGraph] Create graph: 0 [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:30.951.581 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:2697] ConstructOutput] Output:@_anonymous__1:CNode_2{[0]: ValueNode InitDataSetQueue} [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.952.101 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:193] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:0 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.952.175 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 8.97 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.952.293 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 87.36 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.952.404 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:154] CommonUnifyMindIR] start common unify mindir opt graph:0 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.952.451 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.952.566 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 108.92 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.952.584 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.952.619 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 29.19 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.952.638 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.952.652 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 14.87 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.952.666 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.952.720 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 51.11 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.952.836 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:79] BackendCommonOptimization] Status record: start common optimization. graph id: 0 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.952.879 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: add_dynamic_shape_attr [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.952.938 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_0_add_dynamic_shape_attr in 53.97 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.952.954 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.953.016 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_1_convert_dynamic_broadcast_to in 57.71 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.953.068 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_2_convert_const_input_to_attr in 32.03 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.953.102 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_3_custom_op_const_input_to_attr in 15.78 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.953.134 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 13.78 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.953.223 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 66.86 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.953.243 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 1.1 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.953.296 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_7_flatten_concat_fission in 35.03 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.953.344 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 28.06 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.953.385 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_9_broadcast_to_fusion in 22.14 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.953.741 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 330.06 us [WARNING] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.953.858 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:92] BackendCommonOptimization] [PROF]backend_common_optimization costs 1020 usec. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.953.875 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:99] BackendCommonOptimization] Status record: end common optimization. graph id: 0 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.105 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_0_renorm_split in 60.43 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.127 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.223 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_1_reduce_axis_update in 90.7 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.243 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim HistogramFixedWidth not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.259 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_2_histogram_fixed_width_fusion in 15.41 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.294 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim ClipByNorm not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.316 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_3_clip_by_norm_fission in 41.17 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.357 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.373 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_4_tensor_arry_add_flow_cond1 in 40.75 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.386 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayGather not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.400 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_5_tensor_arry_add_flow_cond2 in 11.83 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.413 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.427 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_6_ge_tensor_arry_cast_index in 12.92 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.447 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArray not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.462 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_7_tensor_array_prepare in 20.33 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.475 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.532 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_8_space_to_batch_nd_attr_update in 53.36 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.547 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.576 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_9_batch_to_space_nd_attr_update in 26.61 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.590 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: avg_pool_grad_unify_mindir [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.646 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_10_avg_pool_grad_unify_mindir in 52.91 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.712 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_11_adam_weight_decay_unify_mindir in 46.05 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.775 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_12_cdist_fission in 43.69 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.818 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_13_cdist_grad_fission in 23.4 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.909 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_14_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 71.49 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.954.967 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_15_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 37.38 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.016 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_16_sparse_softmax_cross_entropy_with_logits_unify_mindir in 22.59 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.053 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_17_dropout_unify_mindir1 in 18.5 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.069 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.116 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_18_dropoutgrad_unify_mindir in 44.49 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.155 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.170 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_19_neighbor_exchange_unify_mindir in 35.32 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.184 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.197 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_20_neighbor_exchange_v2_unify_mindir in 12.4 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.211 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.224 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_21_neighbor_exchange_v2_grad_unify_mindir in 11.76 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.254 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.268 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_22_all_to_all_unify_mindir in 30.23 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.284 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim QuantDTypeCast not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.300 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_23_quant_dtype_cst_adjust in 17.02 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.316 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim FSEDecode not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.329 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_24_fse_decode_adjust in 13.86 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.362 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.376 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_25_bn_split in 33.15 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.389 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.454 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_26_bn_grad_unify_mindir in 61.84 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.478 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.491 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_27_bn_grad_split in 13.25 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.505 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.518 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_28_batchnorm_to_bninfer in 11.76 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.530 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.543 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_29_batchnormgrad_to_bninfergrad in 11.17 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.555 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.568 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_30_batch_norm_grad_infer_fission in 11.38 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.666 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_31_ascend_mindir_op_adapter in 80.73 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.686 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_32_DropoutGenMask in 1.36 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.745 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_33_lamb_fission_ge in 42 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.807 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_34_print_insert_placeholder_for_tensor_name in 42.39 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.869 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_35_getnext_for_ge in 41.6 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.929 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_36_sync_bn_split in 40.1 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.955.970 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_37_sync_bn_grad_split in 22.47 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.956.032 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_38_adaptive_max_pool2d_ge_fusion in 41.48 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.956.072 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_39_avg_pool_grad_for_ge in 21.97 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.956.092 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:260] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.956.202 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_40_FlashAttentionFusionV1 in 110.24 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.956.220 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:374] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.956.336 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_41_FlashAttentionFusionV1 in 114.35 us [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:30.956.679 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:30.956.701 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:30.956.714 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:132] GetRunMode] RunMode::kKernelMode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.956.948 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 192.64 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.957.154 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_0_seed_adapter in 75.21 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.957.178 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: env_op_attr_update [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.957.261 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_1_env_op_attr_update in 77.69 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.957.311 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_2_process call inline in 26.97 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.957.380 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_3_expander_fallback in 48.1 us [WARNING] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.957.464 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:139] GEBackendOptimizeACL] [PROF]ascend_backend_optimize_acl costs 401 usec. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:30.957.500 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] InitDataSetQueue is not defined in opdef. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.957.937 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 27.69 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.958.024 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 53.2 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.958.090 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 37.84 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.958.170 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 57.29 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.958.211 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_4_insert_type_transform_op in 19.1 us [WARNING] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.958.294 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:179] GEBackendOptimizeACLAfterKernelSelect] [PROF]ascend_backend_optimize_acl_after_kernel_select costs 466 usec. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.958.354 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_0_DropoutGenMask in 0.54 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.958.418 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_1_cse in 36 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.958.477 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_redundant_op in 38.57 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.958.503 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_3_eliminate_maketuple_getitem in 7.16 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.958.521 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_4_MergeTransData in 1 us [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:30.958.650 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive InitDataSetQueue [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:30.958.774 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive InitDataSetQueue [WARNING] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:30.958.792 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:262] CreateKernel] [PROF]create_kernel costs 153 usec. [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:30.958.839 [mindspore/ccsrc/backend/common/session/session_basic.cc:1140] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 0 start [INFO] DEBUG(164039,ffff87fd6440,python):2024-01-10-11:37:30.958.855 [mindspore/ccsrc/debug/summary/summary.cc:33] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 0 start [INFO] DEBUG(164039,ffff87fd6440,python):2024-01-10-11:37:30.958.870 [mindspore/ccsrc/debug/summary/summary.cc:54] RecurseSetSummaryNodesForAllGraphs] The total summary nodes is: 0 for graph: 0 [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:30.958.995 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:891] PrintGraphExecuteOrder] Graph 0 execution order: [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.042 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[0], node name[Default/InitDataSetQueue-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_0:CNode_2{[0]: ValueNode InitDataSetQueue}] [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.086 [mindspore/ccsrc/backend/common/somas/somas.cc:143] Assign] Start Somas Assign for graph 0 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.113 [mindspore/ccsrc/backend/common/somas/somas.cc:151] Assign] Somas Configure success, configuration info: Device Name: Ascend Run by execution order: 1 Enable debug log: 0 Debug log path: . [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.125 [mindspore/ccsrc/backend/common/somas/somas.cc:154] Assign] Start Initialize SOMAS Model [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.182 [mindspore/ccsrc/backend/common/somas/somas.cc:1065] GraphOutputProcess] Set 0 graph output tensors' aligned size to 0. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.224 [mindspore/ccsrc/backend/common/somas/somas.cc:1135] RefNodeProcess] Special Tensor total size: RefNode: input 0 output 0 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.245 [mindspore/ccsrc/backend/common/somas/somas.cc:551] InitSomasModel] No Tensor from graph 0 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.257 [mindspore/ccsrc/backend/common/somas/somas.cc:157] Assign] End Initialize SOMAS Model [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.267 [mindspore/ccsrc/backend/common/somas/somas.cc:160] Assign] No Somas Tensor in graph 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.278 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:310] DoSomas] Somas allocate success for graph 0 somas size: 0 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.314 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:801] CreateDeviceAddress] Status record: start create device address. graph id: 0 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.390 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:808] CreateDeviceAddress] Status record: end create device address. graph id: 0 [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.423 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1129] CacheGraphOutputToFrontNodeWithIndex] Get graph backend output nodes. [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.447 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1137] CacheGraphOutputToFrontNodeWithIndex] Get graph front output nodes. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.464 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:507] CompileGraph] Status record: end compile graph. graph id: 0 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.546 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:805] CompileGraphFromSegment] Compile cut segment, the cut node: @_anonymous__1:CNode_3{[0]: ValueNode Return, [1]: CNode_2} [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.628 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:521] Transform] Graph(kernel_graph_0) transforms actor begin, strategy:pipeline_with_execution_order [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.732 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1232] BuildLoopCountActor] Create loop count actor: kernel_graph_0_LoopCountActor [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.751 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1265] BuildOutputActor] Create output actor: kernel_graph_0_OutputActor [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.767 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1285] BuildDataPrepareActor] Create data prepare actor: kernel_graph_0_DataPrepareActor [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.796 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1518] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_0 start. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.813 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1520] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_0 end. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.903 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 1_actor_set_kernel_graph_0_memory_actor_insert in 1.55 us [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.924 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 2_actor_set_kernel_graph_0_invalid_data_arrow_elimination in 0.910004 us [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.956 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 3_actor_set_kernel_graph_0_multi_actor_fusion in 16.45 us [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.971 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 4_actor_set_kernel_graph_0_batch_data_arrow_fusion in 0.780004 us [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:30.959.988 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:554] Transform] Graph(kernel_graph_0) transforms actor end. [WARNING] VM(164039,ffff87fd6440,python):2024-01-10-11:37:30.960.048 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:612] CompileGraphs] [PROF]compile_func_graph costs 9968 usec. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:37:30.960.076 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:615] CompileGraphs] Status record: end compile function graph: _anonymous__1, produce actor: kernel_graph_0 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:37:30.960.101 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_0 [INFO] GE(164039,python):2024-01-10-11:37:30.960.611 [scalable_config.cc:55][EVENT]167202 ScalableConfig:device total max size: 34359738368, page_mem_size_total_thresold: 32641751449, uncacheable_size_threshold: 17179869184 [INFO] GE(164039,python):2024-01-10-11:37:31.043.591 [graph_var_manager.cc:1424][EVENT]167202 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:31.043.717 [graph_manager.cc:1248][EVENT]167202 PreRun:PreRun start: graph node size 1, session id 1, graph id 0, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:31.044.648 [atrace_api.c:28](tid:167202) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:31.044.725 [trace_rb_log.c:84](tid:167202) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:31.044.741 [atrace_api.c:32](tid:167202) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:31.044.777 [client_manager.cpp:157][SetProfilingCallback][tid:167202] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:31.045.951 [parallel_partitioner.cc:165][EVENT]167202 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [34] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.045.997 [parallel_partitioner.cc:178][EVENT]167202 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.046.056 [graph_prepare.cc:1378][EVENT]167202 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.046.726 [graph_manager.cc:1050][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [693] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.046.753 [graph_manager.cc:1052][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.046.850 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [1] [INFO] GE(164039,python):2024-01-10-11:37:31.046.881 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.047.020 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [127] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.047.034 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.047.162 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [46] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.047.175 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.047.187 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.047.286 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.047.312 [graph_manager.cc:1054][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [545] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.054.875 [graph_manager.cc:1055][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [7538] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.055.713 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:31.055.737 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.055.748 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [16] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.055.758 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferShapePass is [127] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.055.767 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [44] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.055.776 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:31.055.784 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.055.793 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.055.801 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.056.750 [graph_manager.cc:1056][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [1840] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.056.805 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.056.822 [graph_prepare.cc:1982][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [42] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.056.978 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:31.056.994 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [0] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.004 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.013 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferShapePass is [50] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.022 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [4] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.031 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:31.057.039 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [0] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.047 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.056 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferValuePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.121 [graph_prepare.cc:1983][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [286] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.057.144 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.057.155 [graph_prepare.cc:1984][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.057.169 [graph_prepare.cc:1985][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.057.188 [graph_prepare.cc:1986][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.057.200 [graph_prepare.cc:1987][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.057.214 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.057.226 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.057.239 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.057.307 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.318 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.327 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.336 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.344 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.353 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.361 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.369 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.378 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.386 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.394 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.402 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.411 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.419 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.439 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.448 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.057.470 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.057.483 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.057.510 [graph_prepare.cc:1988][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [302] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.057.523 [graph_manager.cc:1065][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [745] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.070.031 [graph_manager.cc:1077][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12489] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.070.077 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.070.102 [graph_manager.cc:1080][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.072.594 [graph_manager.cc:1081][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [2477] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.072.631 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.072.646 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.072.658 [graph_manager.cc:1082][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.072.687 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.072.700 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.072.713 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.072.829 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [107] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.072.845 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.072.859 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.072.872 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.072.905 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.072.940 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.072.960 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.072.978 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.018 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [30] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.030 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.039 [graph_manager.cc:2700][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [357] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.208 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of EnterPass is [0] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.073.222 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AddNPass is [0] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.073.232 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.073.241 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.073.250 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.073.259 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CastRemovePass is [54] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.073.267 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.073.275 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [24] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.073.284 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [20] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.073.292 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.073.301 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [4] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.073.309 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [0] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.073.317 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.073.325 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.073.333 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.073.343 [graph_manager.cc:2741][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [286] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.352 [graph_manager.cc:2752][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.380 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.391 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.405 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.420 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.431 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.443 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.483 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.497 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.509 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.521 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.536 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.548 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.560 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.571 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.580 [graph_manager.cc:2810][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [205] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.603 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.073.614 [graph_manager.cc:2821][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [25] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.642 [graph_manager.cc:1087][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [967] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.782 [graph_manager.cc:1088][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [127] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.834 [graph_manager.cc:1089][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.853 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.073.866 [graph_manager.cc:1097][EVENT]167202 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:31.073.892 [graph_manager.cc:3325][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.074.076 [engine_place.cc:144][EVENT]167202 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.074.090 [engine_place.cc:144][EVENT]167202 Run:The time cost of aicpu_ascend_kernel::CheckSupported is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.074.172 [graph_manager.cc:3351][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [266] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.074.189 [graph_manager.cc:3364][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.074.256 [engine_partitioner.cc:1139][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.074.272 [engine_partitioner.cc:1142][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.074.389 [engine_partitioner.cc:1148][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [107] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.074.415 [engine_partitioner.cc:1155][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.074.460 [engine_partitioner.cc:1164][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [34] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.074.492 [graph_manager.cc:3405][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [291] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.074.508 [graph_manager.cc:3412][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.189 [graph_manager.cc:3422][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [1666] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.219 [graph_manager.cc:3428][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.346 [graph_manager.cc:3467][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [108] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.363 [graph_manager.cc:3377][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [2163] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.378 [graph_manager.cc:1106][EVENT]167202 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [2491] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.390 [graph_manager.cc:1115][EVENT]167202 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:31.076.411 [graph_manager.cc:1130][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.443 [graph_manager.cc:1131][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.501 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [34] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.519 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.529 [graph_manager.cc:2837][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [63] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.579 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.076.591 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.076.600 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.076.608 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.076.617 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.076.625 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:31.076.635 [graph_manager.cc:2864][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [89] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.646 [graph_manager.cc:2872][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.665 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.679 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.693 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.706 [compile_nodes_pass.cc:88][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.716 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.742 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.777 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [25] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.823 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.838 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.851 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.864 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.878 [graph_manager.cc:2927][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [216] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.918 [graph_manager.cc:2937][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.076.996 [graph_manager.cc:2943][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [66] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.077.011 [graph_manager.cc:2950][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.087.253 [graph_manager.cc:2958][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.087.295 [graph_manager.cc:1132][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [10830] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.087.400 [graph_manager.cc:1135][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [90] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.087.448 [graph_manager.cc:2975][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [28] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.087.613 [graph_manager.cc:2981][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [151] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.087.631 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.087.642 [graph_manager.cc:2986][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.087.651 [graph_manager.cc:1136][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [232] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.087.758 [graph_manager.cc:3555][EVENT]167202 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [63] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.087.849 [engine_partitioner.cc:1139][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.087.865 [engine_partitioner.cc:1142][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.087.937 [engine_partitioner.cc:1148][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [62] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.087.960 [engine_partitioner.cc:1155][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.087.991 [engine_partitioner.cc:1164][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.088.011 [graph_builder.cc:865][EVENT]167202 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [189] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.088.080 [graph_builder.cc:288][EVENT]167202 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [48] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.088.239 [graph_builder.cc:293][EVENT]167202 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [139] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.088.518 [model_builder.cc:1133][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [165] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.088.802 [block_mem_assigner.cc:4069][EVENT]167306 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164039,python):2024-01-10-11:37:31.088.802 [block_mem_assigner.cc:4069][EVENT]167307 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164039,python):2024-01-10-11:37:31.089.131 [model_builder.cc:1144][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [590] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.089.157 [model_builder.cc:1152][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.089.172 [model_builder.cc:1157][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.089.330 [model_builder.cc:1167][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [146] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.089.349 [model_builder.cc:1174][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.089.371 [model_builder.cc:1180][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.089.435 [model_builder.cc:1184][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [53] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.089.455 [graph_builder.cc:304][EVENT]167202 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [1187] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:31.089.722 [logger.cc:1071] 167202 ModelBindStream: model_id=576, stream_id=1857, flag=0. [INFO] GE(164039,python):2024-01-10-11:37:31.089.802 [task_generator.cc:804][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.089.889 [task_generator.cc:805][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [73] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.090.465 [task_generator.cc:814][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [551] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.090.479 [task_generator.cc:954][EVENT]167202 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [683] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.090.548 [task_generator.cc:967][EVENT]167202 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [38] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:31.090.568 [logger.cc:1084] 167202 ModelUnbindStream: model_id=576, stream_id=1857, [INFO] GE(164039,python):2024-01-10-11:37:31.090.625 [graph_builder.cc:310][EVENT]167202 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [1156] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.090.747 [graph_manager.cc:1152][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3078] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.090.765 [graph_manager.cc:1164][EVENT]167202 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:31.090.802 [graph_manager.cc:1271][EVENT]167202 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [45040] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.090.819 [graph_manager.cc:1272][EVENT]167202 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:37:31.091.130 [atrace_api.c:93](tid:167202) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:37:31.091.151 [atrace_api.c:95](tid:167202) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:37:31.091.777 [model_introduction.cc:236][EVENT]167202 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(164039,python):2024-01-10-11:37:31.091.797 [model_introduction.cc:294][EVENT]167202 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(164039,python):2024-01-10-11:37:31.091.809 [model_introduction.cc:366][EVENT]167202 Data:model io_info size:0 [INFO] GE(164039,python):2024-01-10-11:37:31.094.218 [graph_converter.cc:838][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [739] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.094.279 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.094.501 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [207] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.094.558 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.094.571 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [53] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.094.593 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.094.616 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.094.634 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.094.667 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.094.708 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [30] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.094.717 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [40] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.094.735 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.094.752 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.094.764 [graph_converter.cc:849][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [511] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.094.879 [graph_converter.cc:853][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [105] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.095.318 [graph_converter.cc:857][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [428] micro second. [INFO] GE(164039,python):2024-01-10-11:37:31.095.409 [graph_converter.cc:862][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [71] micro second. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:31.096.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_0_LoopCountActor) running, loop count: 1, current count: 1, total running count: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:31.097.203 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_0 execution count: 1, execution time: 137.024 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:37:31.097.290 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:31.097.430 [mindspore/ccsrc/runtime/device/kernel_runtime_manager.cc:35] ClearGraphResource] Clear device Ascend_0 graph 0 runtime resource [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:31.100.753 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:198] Compile] Input plan: +-Transfer,send_epoch_end:false,total_batch:2340) | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:31.100.873 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:216] Compile] Plan before optimization: +-Top | +-Transfer,send_epoch_end:false,total_batch:2340) | | +-Repeat(count:1) | | | +-Batch(batch_size:32 drop_remainder:true) | | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:31.100.894 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:60] PrePass] Running pre pass loops. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:31.100.916 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:31.100.951 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:31.101.033 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:31.101.063 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:31.101.079 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:31.101.105 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:31.101.118 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/cache_transform_pass.cc:182] RunOnTree] Pre pass: Cache transform pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:31.101.143 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/cache_transform_pass.cc:199] RunOnTree] Pre pass: Cache transform pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:31.101.155 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:91] PrePass] Pre pass offload complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:31.101.177 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:116] PostPass] Running post pass loops. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:31.101.211 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:135] PostPass] Post passes complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:31.101.249 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:230] Compile] Plan after optimization: +-Top | +-Transfer,send_epoch_end:false,total_batch:2340) | | +-EpochCtrl(epoch:5) | | | +-Batch(batch_size:32 drop_remainder:true) | | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | | +-MnistDataset [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:31.101.968 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_data_queue.cc:227] AscendTdtQueue] Select MBUF channel, the capacity of data queue is: 128 [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:37:31.102.034 [mindspore/ccsrc/minddata/dataset/engine/datasetops/epoch_ctrl_op.cc:25] EpochCtrlOp] Welcome to Epoch Ctrl Op. [INFO] MD(164039,fffda3fff0f0,python):2024-01-10-11:37:31.104.798 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at train-labels-idx1-ubyte. [INFO] MD(164039,fffda3fff0f0,python):2024-01-10-11:37:31.104.843 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at train-images-idx3-ubyte. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.109.095 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:456] SendDataToAscend] Device queue, sending data to Ascend. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.475.175 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:493] GenerateArgumentsKey] Generate a new compile key for new args, key: 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.475.246 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:501] GenerateArgumentsKey] New cached args: [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.476.139 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:978] CompileInner] Start compiling, phase: train.1704857851317627904.281471260670096.0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.476.173 [mindspore/ccsrc/pipeline/jit/ps/event_message_print.cc:37] PrintEventMessage] Start compiling '_DataWrapper.construct' and it will take a while. Please wait... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.487.491 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1659] VmPipeline] This worker is initialized. No need to add worker action. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:37:31.487.581 [mindspore/ccsrc/backend/graph_compiler/transform.cc:573] CreateBackend] CreateBackend is: ge [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.487.607 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.487.621 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEBUG(164039,ffff87fd6440,python):2024-01-10-11:37:31.487.886 [mindspore/ccsrc/debug/debugger/debugger.cc:129] Init] Debugger got device_id: 0 [INFO] DEBUG(164039,ffff87fd6440,python):2024-01-10-11:37:31.487.901 [mindspore/ccsrc/debug/debugger/debugger.cc:131] Init] Debugger got device_target: Ascend [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.487.933 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1311] Run] Pipeline run [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.487.967 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start parse action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.498.904 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end parse action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.498.966 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start symbol_resolve action. [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.518.283 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] _no_sens_impl_4 update var `grads` with node @_no_sens_impl_4:grads{[0]: CNode_5, [1]: grads} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.518.527 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_no_sens_impl_4] Added global python symbol: {F : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.518.996 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] _no_sens_impl_4 update var `loss` with node @_no_sens_impl_4:loss{[0]: CNode_6, [1]: loss, [2]: CNode_7} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.536.291 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] mindspore_nn_optim_momentum_Momentum_construct_8 update var `gradients` with node @mindspore_nn_optim_momentum_Momentum_construct_8:gradients{[0]: CNode_9, [1]: param_gradients} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.536.584 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] mindspore_nn_optim_momentum_Momentum_construct_8 update var `gradients` with node @mindspore_nn_optim_momentum_Momentum_construct_8:gradients{[0]: CNode_10, [1]: gradients} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.536.849 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] mindspore_nn_optim_momentum_Momentum_construct_8 update var `gradients` with node @mindspore_nn_optim_momentum_Momentum_construct_8:gradients{[0]: CNode_11, [1]: gradients} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.537.102 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] mindspore_nn_optim_momentum_Momentum_construct_8 update var `gradients` with node @mindspore_nn_optim_momentum_Momentum_construct_8:gradients{[0]: CNode_12, [1]: gradients} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.537.832 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_13{[0]: CNode_14, [1]: CNode_15, [2]: CNode_16}, block: 0x46ffc500/mindspore_nn_optim_momentum_Momentum_construct_8, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/nn/optim/momentum.py:211/ self.assignadd(self.global_step, self.global_step_increase_tensor)/ [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.538.408 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_optim_momentum_Momentum_construct_8] Added global python symbol: {F : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.538.649 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_optim_momentum_Momentum_construct_8] Added global python symbol: {_momentum_opt : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.539.337 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_17{[0]: ValueNode Depend, [1]: CNode_18, [2]: CNode_19}, state: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_13{[0]: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_14{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.optim.momentum..', [2]: ValueNode assignadd}, [1]: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_15{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.optim.momentum..', [2]: ValueNode global_step}, [2]: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_16{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.optim.momentum..', [2]: ValueNode global_step_increase_tensor}} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.553.650 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20] Added global python symbol: {F : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.553.914 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20] Added global python symbol: {_get_datatype : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.554.410 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20] Added global python symbol: {_cast_datatype : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.554.566 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20] Added global python symbol: {mstype : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.554.834 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20 update var `grads` with node @mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20:grads{[0]: CNode_21, [1]: CNode_22, [2]: param_grads} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.555.497 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20] Added global python symbol: {reduce_opt : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.565.462 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23] Added global python symbol: {_check_is_tensor : Prim[_check_is_tensor]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.565.987 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_24{[0]: CNode_25, [1]: ValueNode logits, [2]: param_logits, [3]: CNode_26}, block: 0x46ffc500/mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/nn/loss/loss.py:777/ _check_is_tensor('logits', logits, self.cls_name)/ [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.566.505 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_27{[0]: CNode_25, [1]: ValueNode labels, [2]: param_labels, [3]: CNode_28}, block: 0x46ffc500/mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/nn/loss/loss.py:778/ _check_is_tensor('labels', labels, self.cls_name)/ [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.567.136 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_29{[0]: ValueNode Depend, [1]: CNode_30, [2]: CNode_31}, state: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_32{[0]: ValueNode MakeTuple, [1]: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_24{[0]: CNode_25, [1]: ValueNode logits, [2]: param_logits, [3]: CNode_26}, [2]: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_27{[0]: CNode_25, [1]: ValueNode labels, [2]: param_labels, [3]: CNode_28}} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.576.130 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_34, [1]: param_x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.576.415 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_35, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.576.676 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_36, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.576.942 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_37, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.577.196 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_38, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.577.451 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_39, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.577.734 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_40, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.578.020 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_41, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.578.287 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_42, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.578.553 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_43, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.578.807 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_44, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.579.064 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_45, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.590.477 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_46] Added global python symbol: {len : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.590.637 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.590.972 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.591.138 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.591.621 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_46:CNode_48{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.591.757 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_47:x{[0]: CNode_49, [1]: param_фx, [2]: CNode_48} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.592.196 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_46:CNode_50{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.592.655 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_47] Added global python symbol: {len : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.592.720 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_46:CNode_51{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.592.824 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_46:x_shape{[0]: CNode_52, [1]: param_x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.592.962 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.593.388 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.593.668 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_47] Added global python symbol: {F : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.593.783 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_46] Added global python symbol: {F : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.593.850 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_46:CNode_53{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.594.181 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.600.757 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_54] Added global python symbol: {len : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.600.917 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.601.238 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.601.401 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.601.885 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_54:CNode_56{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.602.031 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_55:x{[0]: CNode_57, [1]: param_фx, [2]: CNode_56} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.602.460 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_54:CNode_58{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.602.917 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_55] Added global python symbol: {len : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.602.982 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_54:CNode_59{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.603.088 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_54:x_shape{[0]: CNode_60, [1]: param_x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.603.218 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.603.501 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.603.764 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_55] Added global python symbol: {F : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.603.861 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_54] Added global python symbol: {F : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.603.916 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_54:CNode_61{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.604.227 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.607.868 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_62] Added global python symbol: {len : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.608.027 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.608.349 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.608.508 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.608.949 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_62:CNode_64{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.609.083 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_63:x{[0]: CNode_65, [1]: param_фx, [2]: CNode_64} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.609.518 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_62:CNode_66{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.609.976 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_63] Added global python symbol: {len : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.610.041 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_62:CNode_67{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.610.153 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_62:x_shape{[0]: CNode_68, [1]: param_x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.610.284 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.610.573 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.610.839 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_63] Added global python symbol: {F : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.610.928 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_62] Added global python symbol: {F : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.610.982 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_62:CNode_69{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.611.300 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.615.098 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Flatten_construct_70] Added global python symbol: {F : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.628.540 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1261] ParseNameConstant] The NameConstant is bool:False [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.628.836 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:3 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.629.087 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.629.219 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1261] ParseNameConstant] The NameConstant is bool:True [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.629.951 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_pooling_MaxPool2d_construct_71] Added global python symbol: {isinstance : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.630.049 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_pooling_MaxPool2d_construct_72] Added global python symbol: {isinstance : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.630.103 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_pooling_MaxPool2d_construct_71 update var `isinstance` with node @mindspore_nn_layer_pooling_MaxPool2d_construct_72:CNode_73{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.pooling', [2]: ValueNode isinstance} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.630.260 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_pooling_MaxPool2d_construct_71] Added global python symbol: {tuple : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.630.338 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_pooling_MaxPool2d_construct_72] Added global python symbol: {tuple : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.630.397 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_pooling_MaxPool2d_construct_71 update var `tuple` with node @mindspore_nn_layer_pooling_MaxPool2d_construct_72:CNode_74{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.pooling', [2]: ValueNode tuple} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.630.767 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.630.881 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.631.082 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.631.189 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.631.470 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.672.220 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.672.405 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.673.073 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @canonicalize_axis_75:CNode_76{[0]: ValueNode check_axis_valid_77, [1]: param_axis, [2]: ndim}, block: 0x4ee0c670/canonicalize_axis_75, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1606/ check_axis_valid(axis, ndim)/ [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.673.236 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.673.531 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @canonicalize_axis_75:CNode_78{[0]: ValueNode Depend, [1]: CNode_79, [2]: CNode_80}, state: @canonicalize_axis_75:CNode_76{[0]: ValueNode check_axis_valid_77, [1]: @canonicalize_axis_75:param_axis, [2]: @canonicalize_axis_75:ndim{[0]: CNode_81}} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.673.843 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {isinstance : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.673.997 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {Tensor : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.674.551 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {int : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.675.003 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {bool : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.675.666 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {check_flatten_order_const : Prim[check_flatten_order]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.676.259 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @2↓flatten_83:CNode_84{[0]: CNode_85, [1]: param_order}, block: 0x4ee341f0/2↓flatten_83, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1615/ check_flatten_order_const(order)/ [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.676.693 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {rank_ : Prim[Rank]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.677.045 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.677.103 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.677.322 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {reshape_ : Prim[Reshape]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.677.501 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.677.824 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {ops : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.678.048 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.678.562 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {transpose_ : Prim[Transpose]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.678.967 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_86] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.679.072 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.679.137 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `shape_` with node @flatten_82:CNode_87{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode shape_} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.679.462 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_86] Added global python symbol: {rank_ : Prim[Rank]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.679.530 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `rank_` with node @flatten_82:CNode_88{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode rank_} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.679.826 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `start_dim` with node @flatten_82:param_start_dim [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.679.974 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.680.120 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `end_dim` with node @flatten_82:param_end_dim [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.680.229 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.680.530 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.680.584 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.680.835 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_86] Added global python symbol: {reshape_ : Prim[Reshape]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.680.909 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `reshape_` with node @flatten_82:CNode_89{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode reshape_} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.681.123 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.681.433 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_86] Added global python symbol: {flatten_ : Prim[Flatten]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.681.549 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {flatten_ : Prim[Flatten]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.681.620 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `flatten_` with node @flatten_82:CNode_90{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode flatten_} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.682.010 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `canonicalize_axis` with node ValueNode canonicalize_axis_75 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.682.474 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `check_dim_valid` with node ValueNode check_dim_valid_91 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.683.037 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @4↓flatten_92:CNode_93{[0]: ValueNode check_dim_valid_91, [1]: start_dim, [2]: end_dim}, block: 0x4ee605c0/4↓flatten_92, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1636/ check_dim_valid(start_dim, end_dim)/ [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.683.272 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.683.329 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.683.592 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.684.028 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.684.557 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.685.130 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.685.590 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @2↓flatten_83:CNode_94{[0]: ValueNode Depend, [1]: CNode_95, [2]: CNode_96}, state: @2↓flatten_83:CNode_84{[0]: @flatten_82:CNode_85{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode check_flatten_order_const}, [1]: @flatten_82:param_order} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.685.717 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @4↓flatten_92:CNode_97{[0]: ValueNode Depend, [1]: CNode_98, [2]: CNode_99}, state: @4↓flatten_92:CNode_93{[0]: ValueNode check_dim_valid_91, [1]: @4↓flatten_92:idx{[0]: ValueNode canonicalize_axis_75, [1]: param_start_dim, [2]: x_rank}, [2]: @4↓flatten_92:end_dim{[0]: ValueNode canonicalize_axis_75, [1]: param_end_dim, [2]: x_rank}} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.685.841 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.685.937 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.687.172 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:174] CheckFuncReturn] Function must has 'return' statement, but missing in function 'flatten' at /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1557. FuncGraph: ↓check_dim_valid_100. We will add a 'return None' statement automatically. [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.687.280 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:174] CheckFuncReturn] Function must has 'return' statement, but missing in function 'flatten' at /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1557. FuncGraph: ↓check_axis_valid_101. We will add a 'return None' statement automatically. [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.730.011 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [shape_102] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.738.305 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end symbol_resolve action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.738.362 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start graph_reusing action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.738.379 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.basic.DenseDense[True, None]_ID [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.738.394 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.conv.Conv2dConv2dodict_values([6, 16, 5, 1, 'valid', 0, False, ])_ID [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.738.406 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.conv.Conv2dConv2dodict_values([1, 6, 5, 1, 'valid', 0, False, ])_ID [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.738.421 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end graph_reusing action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.738.439 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start meta_unpack_prepare action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.739.510 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end meta_unpack_prepare action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.739.551 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pre_cconv action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.739.569 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pre_cconv action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.739.588 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start abstract_specialize action. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.760.407 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:502] SendDataToAscend] Begin to send data to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.760.483 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:1182] PrintBeginInfoWhenFirstBatch] Loading dataset and begin to push first batch into device ... [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.762.049 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_106{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.762.074 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:1192] PrintEndInfoWhenFirstBatch] Loading dataset and push first batch into device successful. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.762.120 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.762.134 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.762.650 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.763.068 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 3 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.763.240 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_108{[0]: CNode_109}, new_node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_110{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.763.297 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_108{[0]: CNode_109}, new node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_108{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.763.976 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 4 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.764.982 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 5 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.766.144 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_equal_scalar_112] Added global python symbol: {F : } [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.766.239 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 6 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.766.536 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 4, Shape: NoShape), AbstractScalar(Type: Int64, Value: 3, Shape: NoShape)}, g: _equal_scalar_112 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.767.328 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_113:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_113:CNode_115{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.767.392 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_113:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_113:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.768.165 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 7 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.768.554 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 8 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.769.499 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 9 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.770.618 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 10 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.770.823 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_117{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.770.885 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.771.212 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_118{[0]: CNode_119}, new_node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_120{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.771.262 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_118{[0]: CNode_119}, new node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_118{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.771.667 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 11 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.771.899 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_121:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_121:CNode_122{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.771.962 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_121:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_121:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.772.709 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 12 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.773.810 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 13 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.774.911 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 14 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.775.991 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 15 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.777.107 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 16 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.778.152 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 17 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.779.030 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_logical_not_scala_124] Added global python symbol: {auto_generate : } [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.779.235 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 18 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.779.486 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Bool, Value: true, Shape: NoShape)}, g: _logical_not_scala_124 [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.780.346 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 19 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.781.470 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 20 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.782.461 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 21 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.783.521 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 22 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.784.597 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bool_not_125] Added global python symbol: {_get_cache_prim : } [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.784.606 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 23 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.784.744 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bool_not_125] Added global python symbol: {BoolNot : } [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.785.619 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 24 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.786.712 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 25 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.787.810 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 26 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.788.839 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 27 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.790.066 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 28 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.791.004 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 29 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.792.051 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 30 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.793.115 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 31 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.794.291 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 32 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.795.297 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 33 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.796.326 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 34 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.797.433 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 35 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.798.218 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_126] Added global python symbol: {str : } [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.798.456 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 36 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.798.673 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↻_get_cache_prim_for_pynative_127] Added global python symbol: {str : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.798.947 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↻_get_cache_prim_for_pynative_127 update var `str` with node @↵_get_cache_prim_for_pynative_128:param_фstr [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.799.160 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_126] Added global python symbol: {tuple : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.799.355 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] _get_cache_prim_for_pynative_129 update var `key` with node @_get_cache_prim_for_pynative_129:key{[0]: CNode_130, [1]: key, [2]: CNode_131} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.799.543 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 37 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.800.126 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_132] Added global python symbol: {str : } [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.800.597 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 38 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.800.719 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_132] Added global python symbol: {_PRIM_CACHE : {}} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.800.805 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_126] Added global python symbol: {_PRIM_CACHE : {}} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.801.053 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_132] Added global python symbol: {Primitive : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.801.135 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_126] Added global python symbol: {Primitive : } [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.801.700 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 39 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.801.840 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @✓↓_get_cache_prim_for_pynative_133:CNode_134{[0]: ValueNode MetaFuncGraph-unpack_call.135, [1]: CNode_136, [2]: param_фargs, [3]: param_фkwargs}, block: 0x4f6f4610/✓↓_get_cache_prim_for_pynative_133, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/_primitive_cache.py:84/ prim.__init__(*args, **kwargs)/ [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.802.597 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 2↓_get_cache_prim_for_pynative_137 update var `key` with node @↓_get_cache_prim_for_pynative_138:key{[0]: param_фstr, [1]: param_фkey} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.802.769 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @↻_get_cache_prim_for_pynative_139:CNode_140{[0]: ValueNode Depend, [1]: CNode_141, [2]: CNode_142}, state: @↻_get_cache_prim_for_pynative_139:CNode_143{[0]: ValueNode MetaFuncGraph-add.144, [1]: @↵_get_cache_prim_for_pynative_132:param_@CNode_143, [2]: ValueNode 1} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.802.848 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 40 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.802.868 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @✓↓_get_cache_prim_for_pynative_133:CNode_145{[0]: ValueNode Depend, [1]: CNode_146, [2]: CNode_147}, state: @✓↓_get_cache_prim_for_pynative_133:CNode_134{[0]: ValueNode MetaFuncGraph-unpack_call.135, [1]: @✓↓_get_cache_prim_for_pynative_133:CNode_136{[0]: ValueNode getattr, [1]: prim, [2]: ValueNode __init__}, [2]: @↵_get_cache_prim_for_pynative_132:param_фargs, [3]: @↵_get_cache_prim_for_pynative_132:param_фkwargs} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.803.905 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 41 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.804.202 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_148:CNode_149{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.804.261 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_148:CNode_150{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.804.303 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_148:CNode_151{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.805.036 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BoolNot. node: @bool_not_125:CNode_152{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x}, new_node: @bool_not_125:CNode_153{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.805.071 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 42 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.805.092 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BoolNot. node: @bool_not_125:CNode_152{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x}, new node: @bool_not_125:CNode_152{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.806.177 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 43 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.807.212 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 44 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.808.270 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 45 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.809.422 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 46 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.810.529 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 47 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.811.537 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 48 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.811.740 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_equal_string_154] Added global python symbol: {F : } [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.812.101 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: String, Value: C, Shape: NoShape), AbstractScalar(Type: String, Value: F, Shape: NoShape)}, g: _equal_string_154 [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.812.679 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 49 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.813.650 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_155:CNode_156{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_155:CNode_157{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.813.735 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_155:CNode_156{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_155:CNode_156{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.813.829 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 50 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.814.848 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 51 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.815.837 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 52 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.816.306 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_neg_scalar_159] Added global python symbol: {F : } [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.816.638 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 1, Shape: NoShape)}, g: _neg_scalar_159 [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.816.996 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 53 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.817.292 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarUsub. node: @_neg_scalar_160:CNode_161{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x}, new_node: @_neg_scalar_160:CNode_162{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.817.347 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarUsub. node: @_neg_scalar_160:CNode_161{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x}, new node: @_neg_scalar_160:CNode_161{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.818.066 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 54 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.818.084 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_163:CNode_164{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_163:CNode_165{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.818.147 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_163:CNode_164{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_163:CNode_164{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.818.595 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @3↓flatten_166:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput}, new_node: @3↓flatten_166:CNode_167{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.818.658 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @3↓flatten_166:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput}, new node: @3↓flatten_166:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.819.075 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 55 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.820.151 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 56 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.821.281 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 57 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.822.371 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 58 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.822.751 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_169] Added global python symbol: {F : } [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.823.356 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 59 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.823.446 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_169] Added global python symbol: {InSequence : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.823.822 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_169] Added global python symbol: {const_utils : } [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.824.335 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 4, Shape: NoShape), AbstractTuple{ element[0]: AbstractScalar(Type: Int64, Value: 0, Shape: NoShape), element[1]: AbstractScalar(Type: Int64, Value: 1, Shape: NoShape), sequence_nodes: {@✓3↓flatten_170:CNode_171{[0]: ValueNode MakeTuple, [1]: ValueNode 0, [2]: ValueNode 1}, elements_use_flags: {ptr: 0x4f75ad10, value: [const vector]{0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: _number_in_tuple_169 [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.824.495 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 60 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.825.566 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 61 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.826.665 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 62 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.827.765 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 63 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.827.898 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Flatten. node: @↓✓3↓flatten_172:CNode_173{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput}, new_node: @↓✓3↓flatten_172:CNode_174{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.827.963 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Flatten. node: @↓✓3↓flatten_172:CNode_173{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput}, new node: @↓✓3↓flatten_172:CNode_173{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.828.313 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_175:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_175:CNode_176{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.828.363 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_175:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_175:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.828.832 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 64 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.830.082 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 65 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.831.074 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 66 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.831.278 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_not_equal_scalar_178] Added global python symbol: {F : } [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.831.648 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 2, Shape: NoShape), AbstractScalar(Type: Int64, Value: 2, Shape: NoShape)}, g: _not_equal_scalar_178 [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.832.194 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 67 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.832.466 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_179:CNode_180{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_179:CNode_181{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.832.530 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_179:CNode_180{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_179:CNode_180{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.833.250 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 68 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.834.431 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 69 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.834.625 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_183:CNode_184{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_183:CNode_185{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.834.690 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_183:CNode_184{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_183:CNode_184{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.835.433 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 70 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.835.908 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_186:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_186:CNode_187{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias, [3]: ValueNode 0} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.835.977 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_186:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_186:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias, [3]: ValueNode 0} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.836.325 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_188{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.836.378 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.836.538 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 71 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.836.570 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_189:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_189:CNode_190{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.836.617 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_189:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_189:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.837.484 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_191:CNode_192{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_191:CNode_193{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.837.546 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_191:CNode_192{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_191:CNode_192{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.837.677 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 72 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.838.802 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 73 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.839.535 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_194:CNode_195{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_194:CNode_196{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.839.598 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_194:CNode_195{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_194:CNode_195{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.839.896 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 74 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.840.808 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_197:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_197:CNode_198{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias, [3]: ValueNode 0} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.840.874 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 75 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.840.882 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_197:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_197:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias, [3]: ValueNode 0} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.841.168 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_199{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.841.219 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.841.408 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_200:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_200:CNode_201{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.841.454 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_200:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_200:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.842.082 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 76 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.842.372 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_202:CNode_203{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_202:CNode_204{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.842.435 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_202:CNode_203{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_202:CNode_203{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.843.120 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 77 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.844.112 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 78 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.844.267 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_205:CNode_206{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_205:CNode_207{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.844.330 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_205:CNode_206{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_205:CNode_206{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.845.194 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 79 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.845.530 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_208:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_208:CNode_209{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias, [3]: ValueNode 0} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.845.597 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_208:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_208:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias, [3]: ValueNode 0} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.846.277 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 80 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.847.332 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 81 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.848.400 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 82 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.848.550 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny)}, g: hyper_map_212 [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.849.549 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 83 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.850.705 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 84 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.851.045 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_ones_like_tensor_213] Added global python symbol: {P : } [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.851.410 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny)}, g: _ones_like_tensor_213 [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.851.727 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 85 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.852.547 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: OnesLike. node: @_ones_like_tensor_214:grads{[0]: ValueNode PrimFunc_OnesLike, [1]: param_x}, new_node: @_ones_like_tensor_214:CNode_215{[0]: ValueNode PrimFunc_OnesLike, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.852.610 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_OnesLike. node: @_ones_like_tensor_214:grads{[0]: ValueNode PrimFunc_OnesLike, [1]: param_x}, new node: @_ones_like_tensor_214:grads{[0]: ValueNode PrimFunc_OnesLike, [1]: param_x} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.852.777 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 86 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.853.014 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: env_get(AbstractScalar(Type: Object:EnvType, Value: ValueAny, Shape: NoShape), )}, AbstractTuple{ element[0]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[1]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[2]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[3]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[4]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[5]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[6]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[7]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), sequence_nodes: {@_no_sens_impl_216:CNode_217{[0]: ValueNode MakeTuple, [1]: param_conv1.weight, [2]: param_conv2.weight, [3]: param_fc1.weight, [4]: param_fc1.bias, [5]: param_fc2.weight, [6]: param_fc2.bias, [7]: param_fc3.weight, [8]: param_fc3.bias}, elements_use_flags: {ptr: 0x4f843d20, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: hyper_map_218 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.853.391 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: env_get(AbstractScalar(Type: Object:EnvType, Value: ValueAny, Shape: NoShape), )}, AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny)}, g: hyper_map_219 [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.853.886 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 87 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.855.001 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 88 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.856.022 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 89 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.856.178 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensor_env_get_221] Added global python symbol: {environ_get : Prim[EnvironGet]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.856.386 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensor_env_get_221] Added global python symbol: {ref_to_embed : Prim[RefToEmbed]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.856.643 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensor_env_get_221] Added global python symbol: {tensor_zeros_like : Prim[ZerosLike]} [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.856.964 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Object:EnvType, Value: ValueAny, Shape: NoShape), AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny)}, g: _tensor_env_get_221 [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.857.112 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 90 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.858.038 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_222:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_222:CNode_223{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.858.104 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_222:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_222:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.858.398 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 91 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.859.266 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_224:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_224:CNode_225{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.859.327 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_224:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_224:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.859.376 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 92 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.860.410 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_226:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_226:CNode_227{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.860.471 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_226:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_226:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.860.492 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 93 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.861.539 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_228:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_228:CNode_229{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.861.555 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 94 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.861.620 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_228:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_228:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.862.640 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 95 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.862.774 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_230:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_230:CNode_231{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.862.837 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_230:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_230:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.863.736 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 96 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.863.896 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_232:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_232:CNode_233{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.863.959 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_232:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_232:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.864.789 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 97 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.865.041 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_234:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_234:CNode_235{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.865.103 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_234:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_234:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.865.947 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 98 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.866.208 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_236:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_236:CNode_237{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.866.270 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_236:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_236:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.866.976 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 99 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.867.174 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: S-signature()}, AbstractTuple{ element[0]: AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[1]: AbstractTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[2]: AbstractTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[3]: AbstractTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[4]: AbstractTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[5]: AbstractTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[6]: AbstractTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[7]: AbstractTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), sequence_nodes: {@hyper_map_239:grads{[0]: ValueNode MakeTuple, [1]: grads, [2]: grads, [3]: grads, [4]: grads, [5]: grads, [6]: grads, [7]: grads, [8]: grads}, elements_use_flags: {ptr: 0x4f8e5c60, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: map_240 [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:31.868.065 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 100 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.870.091 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensors_get_datatype_242] Added global python symbol: {F : } [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.870.435 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny)}, g: _tensors_get_datatype_242 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.879.455 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensors_cast_datatype_244] Added global python symbol: {F : } [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.879.829 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractType(Value: Float32), AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny)}, g: _tensors_cast_datatype_244 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.887.450 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: S-signature(AbstractTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x496a9340, value: Tensor(shape=[], dtype=Float32, value=0.25)), AbstractScalar(Type: Bool, Value: true, Shape: NoShape), Prim: S_Prim_AllGather, Prim: S_Prim_AllReduce, )}, AbstractTuple{ element[0]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[1]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[2]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[3]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[4]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[5]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[6]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[7]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), sequence_nodes: {ValueNode (true, true, true, true, true, true, true, true), elements_use_flags: {ptr: 0x4f9d2e80, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} , AbstractTuple{ element[0]: AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[1]: AbstractTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[2]: AbstractTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[3]: AbstractTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[4]: AbstractTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[5]: AbstractTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[6]: AbstractTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[7]: AbstractTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), sequence_nodes: {@map_245:grads{[0]: ValueNode MakeTuple, [1]: grads, [2]: grads, [3]: grads, [4]: grads, [5]: grads, [6]: grads, [7]: grads, [8]: grads}, elements_use_flags: {ptr: 0x4f9b1800, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: map_246 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.893.894 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensors_allreduce_248] Added global python symbol: {F : } [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.894.981 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x496a9340, value: Tensor(shape=[], dtype=Float32, value=0.25)), AbstractScalar(Type: Bool, Value: true, Shape: NoShape), Prim: S_Prim_AllGather, Prim: S_Prim_AllReduce, AbstractScalar(Type: Bool, Value: true, Shape: NoShape), AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny)}, g: _tensors_allreduce_248 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.897.906 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_249:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_249:CNode_250{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.897.983 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_249:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_249:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.900.316 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_251:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_251:CNode_252{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.900.392 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_251:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_251:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.902.619 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_253:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_253:CNode_254{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.902.697 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_253:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_253:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.904.901 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_255:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_255:CNode_256{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.904.976 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_255:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_255:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.907.199 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_257:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_257:CNode_258{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.907.288 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_257:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_257:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.909.535 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_259:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_259:CNode_260{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.909.608 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_259:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_259:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.912.014 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_261:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_261:CNode_262{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.912.089 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_261:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_261:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.914.326 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_263:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_263:CNode_264{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.914.404 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_263:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_263:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.915.130 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: S-signature()}, AbstractTuple{ element[0]: AbstractType(Value: Float32), element[1]: AbstractType(Value: Float32), element[2]: AbstractType(Value: Float32), element[3]: AbstractType(Value: Float32), element[4]: AbstractType(Value: Float32), element[5]: AbstractType(Value: Float32), element[6]: AbstractType(Value: Float32), element[7]: AbstractType(Value: Float32), sequence_nodes: {@map_265:datatypes{[0]: ValueNode MakeTuple, [1]: datatypes, [2]: datatypes, [3]: datatypes, [4]: datatypes, [5]: datatypes, [6]: datatypes, [7]: datatypes, [8]: datatypes}, elements_use_flags: {ptr: 0x4f96df90, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} , AbstractTuple{ element[0]: AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[1]: AbstractTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[2]: AbstractTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[3]: AbstractTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[4]: AbstractTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[5]: AbstractTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[6]: AbstractTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[7]: AbstractTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), sequence_nodes: {@map_266:new_grad{[0]: ValueNode MakeTuple, [1]: new_grad, [2]: new_grad, [3]: new_grad, [4]: new_grad, [5]: new_grad, [6]: new_grad, [7]: new_grad, [8]: new_grad}, elements_use_flags: {ptr: 0x4fb3c240, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: map_267 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.924.930 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: S-signature(Prim: S_Prim_ApplyMomentum, AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), )}, AbstractTuple{ element[0]: AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[1]: AbstractTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[2]: AbstractTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[3]: AbstractTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[4]: AbstractTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[5]: AbstractTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[6]: AbstractTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), element[7]: AbstractTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), sequence_nodes: {@map_270:new_grad{[0]: ValueNode MakeTuple, [1]: new_grad, [2]: new_grad, [3]: new_grad, [4]: new_grad, [5]: new_grad, [6]: new_grad, [7]: new_grad, [8]: new_grad}, elements_use_flags: {ptr: 0x4ac9fcd0, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} , AbstractTuple{ element[0]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[1]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[2]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[3]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[4]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[5]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[6]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[7]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), sequence_nodes: {@mindspore_nn_optim_momentum_Momentum_construct_271:CNode_272{[0]: ValueNode MakeTuple, [1]: param_conv1.weight, [2]: param_conv2.weight, [3]: param_fc1.weight, [4]: param_fc1.bias, [5]: param_fc2.weight, [6]: param_fc2.bias, [7]: param_fc3.weight, [8]: param_fc3.bias}, elements_use_flags: {ptr: 0x4acb5500, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} , AbstractTuple{ element[0]: AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[1]: AbstractRefTensor(key: moments.conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[2]: AbstractRefTensor(key: moments.fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[3]: AbstractRefTensor(key: moments.fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[4]: AbstractRefTensor(key: moments.fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[5]: AbstractRefTensor(key: moments.fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[6]: AbstractRefTensor(key: moments.fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), element[7]: AbstractRefTensor(key: moments.fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), sequence_nodes: {@mindspore_nn_optim_momentum_Momentum_construct_271:CNode_273{[0]: ValueNode MakeTuple, [1]: param_moments.conv1.weight, [2]: param_moments.conv2.weight, [3]: param_moments.fc1.weight, [4]: param_moments.fc1.bias, [5]: param_moments.fc2.weight, [6]: param_moments.fc2.bias, [7]: param_moments.fc3.weight, [8]: param_moments.fc3.bias}, elements_use_flags: {ptr: 0x4acb5da0, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: hyper_map_274 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.925.584 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: S-signature(Prim: S_Prim_ApplyMomentum, AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), )}, AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny)}, g: hyper_map_275 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.929.926 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensor_run_opt_ext_277] Added global python symbol: {F : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.930.141 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1261] ParseNameConstant] The NameConstant is bool:True [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:31.930.615 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{Prim: S_Prim_ApplyMomentum, AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny)}, g: _tensor_run_opt_ext_277 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.938.934 [mindspore/ccsrc/pipeline/jit/ps/action.cc:282] AbstractAnalyze] function call depth: 0, simulate call depth: 0 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.939.143 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:222] Run] Specialize set top func graph context: {FuncGraph: mindspore_train_dataset_helper__DataWrapper_construct_103 Args: [0]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [1]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [2]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [3]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [4]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [5]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [6]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [7]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [8]: AbstractRefTensor(key: global_step ref_value: AbstractRefTensor(shape: (1), element: AbstractScalar(Type: Int32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [9]: AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [10]: AbstractRefTensor(key: moments.conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [11]: AbstractRefTensor(key: moments.fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [12]: AbstractRefTensor(key: moments.fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [13]: AbstractRefTensor(key: moments.fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [14]: AbstractRefTensor(key: moments.fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [15]: AbstractRefTensor(key: moments.fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [16]: AbstractRefTensor(key: moments.fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [17]: AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [18]: AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), Parent: } [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.939.343 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @mindspore_train_dataset_helper__DataWrapper_construct_103:CNode_278{[0]: ValueNode MetaFuncGraph-unpack_call.279, [1]: ValueNode mindspore_nn_wrap_cell_wrapper_TrainOneStepCell_construct_280, [2]: outputs}, flag: 1 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.939.709 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @UnpackCall_281:CNode_278{[0]: param_282, [1]: CNode_278, [2]: CNode_278}, flag: 1 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.940.053 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @mindspore_nn_wrap_cell_wrapper_TrainOneStepCell_construct_283:CNode_284{[0]: ValueNode ✓mindspore_nn_wrap_cell_wrapper_TrainOneStepCell_construct_285}, flag: 1 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.940.284 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @✓mindspore_nn_wrap_cell_wrapper_TrainOneStepCell_construct_285:CNode_286{[0]: ValueNode MetaFuncGraph-unpack_call.287, [1]: ValueNode _no_sens_impl_288, [2]: CNode_289}, flag: 1 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.940.541 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @UnpackCall_290:CNode_286{[0]: param_291, [1]: CNode_286, [2]: CNode_286}, flag: 1 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.940.713 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @_no_sens_impl_216:CNode_292{[0]: ValueNode ✗_no_sens_impl_293}, flag: 1 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.940.808 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @✗_no_sens_impl_293:CNode_294{[0]: ValueNode ↓_no_sens_impl_295}, flag: 1 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.940.891 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @_no_sens_impl_216:loss{[0]: ValueNode S_Prim_Depend, [1]: loss, [2]: CNode_7}, flag: 1 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.940.944 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @_no_sens_impl_216:CNode_7{[0]: ValueNode mindspore_nn_optim_momentum_Momentum_construct_271, [1]: grads}, flag: 1 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:31.946.600 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @mindspore_nn_optim_momentum_Momentum_construct_271:CNode_17{[0]: ValueNode Depend, [1]: CNode_18, [2]: CNode_19}, flag: 1 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.972.754 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end abstract_specialize action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.972.828 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pack_expand action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.973.253 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pack_expand action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.973.295 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start auto_monad action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.976.475 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end auto_monad action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.976.522 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start inline action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.976.539 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end inline action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.976.558 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pre_auto_parallel action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.976.586 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pre_auto_parallel action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.976.603 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pipeline_split action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.976.617 [mindspore/ccsrc/pipeline/jit/ps/pipeline_split.cc:247] PipelineSplit] Only auto_parallel and semi_auto_parallel support pipeline split. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.976.628 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pipeline_split action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:31.976.644 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start optimize action. [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.986.814 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [fill_296] Added global python symbol: {cast_ : Prim[Cast]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.987.059 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] fill_296 update var `value` with node @fill_296:value{[0]: CNode_297, [1]: param_value, [2]: param_type} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:31.987.305 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [fill_296] Added global python symbol: {fillv2_ : Prim[FillV2]} [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.010.476 [mindspore/ccsrc/frontend/parallel/pynative_shard/pynative_shard.cc:334] PynativeShard] Only auto_parallel and semi_auto_parallel support pynative shard [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.010.543 [mindspore/ccsrc/frontend/parallel/step_parallel.cc:3009] StepParallel] Strategies would be ignored in data_parallel, shard() only valid in [semi_]auto_parallel. [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:32.027.070 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bprop_depend_298] Added global python symbol: {C : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:37:32.032.174 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bprop_load_299] Added global python symbol: {C : } [INFO] OPTIMIZER(164039,ffff87fd6440,python):2024-01-10-11:37:32.040.913 [mindspore/ccsrc/frontend/optimizer/ad/bprop_utils.cc:70] GetBprop] Fail to find bprop function for UpdateState. fn: None [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.056.670 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractUMonad(ValueAny)}, g: _zeros_like_u_monad_302 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.070.922 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractUMonad(ValueAny), AbstractUMonad(ValueAny)}, g: hyper_map_303 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.073.477 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractUMonad(ValueAny), AbstractUMonad(ValueAny)}, g: _add_umonad_umonad_304 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.076.171 [mindspore/ccsrc/pipeline/jit/ps/action.cc:282] AbstractAnalyze] function call depth: 0, simulate call depth: 0 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:32.076.366 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:222] Run] Specialize set top func graph context: {FuncGraph: 1_mindspore_train_dataset_helper__DataWrapper_construct_300 Args: [0]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [1]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [2]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [3]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [4]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [5]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [6]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [7]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [8]: AbstractRefTensor(key: global_step ref_value: AbstractRefTensor(shape: (1), element: AbstractScalar(Type: Int32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [9]: AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [10]: AbstractRefTensor(key: moments.conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [11]: AbstractRefTensor(key: moments.fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [12]: AbstractRefTensor(key: moments.fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [13]: AbstractRefTensor(key: moments.fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [14]: AbstractRefTensor(key: moments.fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [15]: AbstractRefTensor(key: moments.fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [16]: AbstractRefTensor(key: moments.fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [17]: AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [18]: AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), Parent: } [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.118.552 [mindspore/ccsrc/frontend/parallel/pynative_shard/pynative_shard.cc:334] PynativeShard] Only auto_parallel and semi_auto_parallel support pynative shard [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.124.118 [mindspore/ccsrc/pipeline/jit/ps/action.cc:282] AbstractAnalyze] function call depth: 0, simulate call depth: 0 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:37:32.124.516 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:222] Run] Specialize set top func graph context: {FuncGraph: 251_1_mindspore_train_dataset_helper__DataWrapper_construct_315 Args: [0]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [1]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [2]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [3]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [4]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [5]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [6]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [7]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [8]: AbstractRefTensor(key: global_step ref_value: AbstractRefTensor(shape: (1), element: AbstractScalar(Type: Int32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [9]: AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [10]: AbstractRefTensor(key: moments.conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [11]: AbstractRefTensor(key: moments.fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [12]: AbstractRefTensor(key: moments.fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [13]: AbstractRefTensor(key: moments.fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [14]: AbstractRefTensor(key: moments.fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [15]: AbstractRefTensor(key: moments.fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [16]: AbstractRefTensor(key: moments.fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [17]: AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [18]: AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), Parent: } [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.131.072 [mindspore/ccsrc/frontend/parallel/pynative_shard/pynative_shard.cc:334] PynativeShard] Only auto_parallel and semi_auto_parallel support pynative shard [INFO] OPTIMIZER(164039,ffff87fd6440,python):2024-01-10-11:37:32.136.316 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] OPTIMIZER(164039,ffff87fd6440,python):2024-01-10-11:37:32.137.356 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] OPTIMIZER(164039,ffff87fd6440,python):2024-01-10-11:37:32.138.142 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.138.289 [mindspore/ccsrc/frontend/parallel/cache_embedding/cache_embedding.cc:702] AddCacheEmbedding] Parameters are all not cache enable. [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.139.270 [mindspore/ccsrc/frontend/parallel/pass/assign_add_opt.cc:120] AssignAddOpt] parallel::g_device_manager is not initialized. [INFO] OPTIMIZER(164039,ffff87fd6440,python):2024-01-10-11:37:32.139.335 [mindspore/ccsrc/frontend/optimizer/comm_op_reuse_tag.cc:59] AddCommOpReuseTag] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.139.355 [mindspore/ccsrc/frontend/parallel/pass/overlap_opt_shard_in_pipeline.cc:70] OverlapOptShardInPipeline] parallel::g_device_manager is not initialized. [INFO] OPTIMIZER(164039,ffff87fd6440,python):2024-01-10-11:37:32.139.375 [mindspore/ccsrc/frontend/optimizer/grouped_pairwise_exchange_alltoall.cc:673] SetGroupedPairwiseExchangeAllToAll] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.139.400 [mindspore/ccsrc/frontend/parallel/pass/overlap_gradmatmul_and_gradallreduce.cc:358] OverlapGradMatmulAndGradAllreduce] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.139.418 [mindspore/ccsrc/frontend/parallel/pass/split_matmul_comm_elementwise_fp.cc:184] SplitMatmulCommElementwiseFp] SplitMatmulCommElementwiseFp is only support under [semi_]auto_parallel, skip it. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.139.467 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end optimize action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.139.491 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start auto_monad_reorder action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.139.832 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end auto_monad_reorder action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.139.866 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start get_jit_bprop_graph action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.139.883 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end get_jit_bprop_graph action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.139.900 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start eliminate_special_op_node action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.140.864 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end eliminate_special_op_node action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.140.908 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start validate action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.141.184 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end validate action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.141.217 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start distribtued_split action. [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.141.240 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:372] GenerateStrategy] Current parallel mode is data_parallel [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.141.255 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:384] GenerateStrategy] Generated distributed strategy is 1 [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.141.623 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:1270] Run] All nodes are on this precoess so there's no need to build and split distributed graph. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.141.658 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end distribtued_split action. [INFO] PROFILER(164039,ffff87fd6440,python):2024-01-10-11:37:32.141.778 [mindspore/ccsrc/plugin/device/ascend/hal/profiler/parallel_strategy_profiling.cc:48] IsProfilingParallelStrategyEnabled] Profiling parallel strategy is disabled. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.141.803 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start task_emit action. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.142.189 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.142.213 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.142.229 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1252] SetRunMode] Run graph mode with kernel by kernel by configuration. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:37:32.142.342 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:546] CompileGraphs] Status record: start compile function graph: 381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.142.537 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unify_mindir_pm_0_erase_invalid_micro_depend in 2.2 us [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.143.640 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.143.669 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.033 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.055 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.085 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.099 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.117 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.134 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.149 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.160 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.178 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.191 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.206 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.217 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.234 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.248 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.262 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.274 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.291 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.303 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.319 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.329 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.345 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.361 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.397 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.414 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.431 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.442 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.457 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.472 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.487 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.498 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.514 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.528 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.543 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.554 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.570 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.584 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.599 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.611 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.625 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.639 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.653 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.665 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.679 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.690 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.708 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.725 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.739 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.754 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.772 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.782 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.797 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.807 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.822 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.833 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.850 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.863 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.882 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.898 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.914 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.925 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.940 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.951 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.969 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.981 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.144.999 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.014 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.029 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.043 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.069 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.085 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.101 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.115 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.134 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.148 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.163 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.178 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.195 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.209 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.224 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.238 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.255 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.267 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.285 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.299 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.314 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.328 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.347 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.361 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.375 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.389 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.407 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.419 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.445 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.459 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.477 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.491 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.509 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.524 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.538 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.552 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.570 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.584 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.599 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.613 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.630 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.645 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.662 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.676 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.724 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.742 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.758 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.774 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.790 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.805 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.823 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.843 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.861 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.875 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.889 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.904 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.921 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.934 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.952 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.965 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.983 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.145.994 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.011 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.026 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.045 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.059 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.076 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.090 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.107 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.121 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.139 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.153 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.171 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.185 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.207 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.222 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.241 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.255 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.272 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.286 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.303 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.317 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.331 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.345 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.364 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.380 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.398 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.410 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.425 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.439 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.453 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.466 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.480 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.496 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.511 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.522 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.536 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.547 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.574 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.585 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.602 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.615 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.632 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.644 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.658 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.669 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.686 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.697 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.711 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.722 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.739 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.750 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.764 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.775 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.793 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.804 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.818 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.829 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.845 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.857 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.872 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.893 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.908 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.919 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.933 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.947 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.961 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.971 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.985 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.146.998 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:37:32.147.077 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:727] CompileGraph] Compile graph: 381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316, Split segments size: 2 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:37:32.147.124 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:762] CompileGraphFromSegment] Compile normal segment, the first node: @381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316:equiv_CNode_317{[0]: ValueNode Load, [1]: param_fc3.bias, [2]: ValueNode U} [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.147.581 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:421] CompileGraph] Status record: start compile graph. [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.147.627 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:1940] ConstructKernelGraph] Create graph: 1 [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.148.390 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:872] CreateNewValueNode] Data sync for Tensor Tensor(shape=[1], dtype=Int32, value=[1]) [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.148.582 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:872] CreateNewValueNode] Data sync for Tensor Tensor(shape=[], dtype=Float32, value=0.25) [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.150.033 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:2697] ConstructOutput] Output:@381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316:CNode_278{[0]: ValueNode Depend, [1]: CNode_278, [2]: CNode_318} [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.151.855 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:193] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:1 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.152.315 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 89.7 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.152.952 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 591.1 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.153.875 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:154] CommonUnifyMindIR] start common unify mindir opt graph:1 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.154.205 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.154.647 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 436.16 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.154.679 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.154.719 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 36.91 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.154.739 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.154.754 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 14.14 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.154.767 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.155.323 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 541.62 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.156.246 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:79] BackendCommonOptimization] Status record: start common optimization. graph id: 1 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.156.577 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: add_dynamic_shape_attr [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.157.389 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_0_add_dynamic_shape_attr in 804.22 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.157.423 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.157.476 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_1_convert_dynamic_broadcast_to in 49.95 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.158.134 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_2_convert_const_input_to_attr in 624.1 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.158.678 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_3_custom_op_const_input_to_attr in 504.77 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.159.143 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 430.19 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.160.604 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 1420.79 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.160.660 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 17.17 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.161.074 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_7_flatten_concat_fission in 380.57 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.161.481 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 359.58 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.161.882 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_9_broadcast_to_fusion in 366.44 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.162.427 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 507.85 us [WARNING] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.163.320 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:92] BackendCommonOptimization] [PROF]backend_common_optimization costs 7075 usec. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.163.352 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:99] BackendCommonOptimization] Status record: end common optimization. graph id: 1 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.164.150 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_0_renorm_split in 387.73 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.164.183 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.165.156 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_1_reduce_axis_update in 961.18 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.165.187 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim HistogramFixedWidth not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.165.210 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_2_histogram_fixed_width_fusion in 23.37 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.165.230 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim ClipByNorm not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.165.247 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_3_clip_by_norm_fission in 17.15 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.165.265 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.165.282 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_4_tensor_arry_add_flow_cond1 in 17 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.165.299 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayGather not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.165.315 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_5_tensor_arry_add_flow_cond2 in 14.92 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.165.328 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.165.344 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_6_ge_tensor_arry_cast_index in 14.84 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.165.358 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArray not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.165.374 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_7_tensor_array_prepare in 15.7 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.165.391 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.165.448 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_8_space_to_batch_nd_attr_update in 52.13 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.165.467 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.165.500 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_9_batch_to_space_nd_attr_update in 30.12 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.165.514 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: avg_pool_grad_unify_mindir [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.165.551 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_10_avg_pool_grad_unify_mindir in 34.12 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.165.986 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_11_adam_weight_decay_unify_mindir in 409.27 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.166.382 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_12_cdist_fission in 363.82 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.166.773 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_13_cdist_grad_fission in 356.12 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.167.278 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_14_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 469.41 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.168.595 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_15_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 1272.99 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.169.069 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_16_sparse_softmax_cross_entropy_with_logits_unify_mindir in 428.08 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.169.502 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_17_dropout_unify_mindir1 in 394.08 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.169.530 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.169.948 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_18_dropoutgrad_unify_mindir in 407.5 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.169.980 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.169.997 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_19_neighbor_exchange_unify_mindir in 17.23 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.011 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.025 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_20_neighbor_exchange_v2_unify_mindir in 12.71 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.039 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.061 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_21_neighbor_exchange_v2_grad_unify_mindir in 19.94 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.075 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.090 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_22_all_to_all_unify_mindir in 13 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.106 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim QuantDTypeCast not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.120 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_23_quant_dtype_cst_adjust in 14.93 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.134 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim FSEDecode not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.148 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_24_fse_decode_adjust in 13.71 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.161 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.174 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_25_bn_split in 12.29 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.186 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.250 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_26_bn_grad_unify_mindir in 59.04 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.268 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.283 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_27_bn_grad_split in 13.13 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.296 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.311 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_28_batchnorm_to_bninfer in 13.64 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.324 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.339 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_29_batchnormgrad_to_bninfergrad in 13.32 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.352 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.170.365 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_30_batch_norm_grad_infer_fission in 11.43 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.171.359 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_31_ascend_mindir_op_adapter in 963.04 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.171.394 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_32_DropoutGenMask in 1.61 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.171.850 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_33_lamb_fission_ge in 419.63 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.172.806 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_34_print_insert_placeholder_for_tensor_name in 921.84 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.173.266 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_35_getnext_for_ge in 423.59 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.173.698 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_36_sync_bn_split in 388.7 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.174.128 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_37_sync_bn_grad_split in 390.38 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.174.545 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_38_adaptive_max_pool2d_ge_fusion in 384.68 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.174.962 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_39_avg_pool_grad_for_ge in 384.28 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.174.990 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:260] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.175.960 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_40_FlashAttentionFusionV1 in 963.89 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.175.992 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:374] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.176.953 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_41_FlashAttentionFusionV1 in 955.53 us [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.179.049 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:37:32.179.081 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.179.097 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:132] GetRunMode] RunMode::kKernelMode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.180.463 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 1315.39 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.182.435 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_0_seed_adapter in 1297.01 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.182.474 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: env_op_attr_update [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.183.307 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_1_env_op_attr_update in 816.74 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.183.766 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_2_process call inline in 413.78 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.184.160 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_3_expander_fallback in 347.37 us [WARNING] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.184.807 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:139] GEBackendOptimizeACL] [PROF]ascend_backend_optimize_acl costs 3688 usec. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.184.867 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] GetNext is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.185.468 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2D is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.185.885 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPool is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.186.068 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2D is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.186.218 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPool is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.186.435 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.186.900 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.187.104 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.187.315 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] SoftmaxCrossEntropyWithLogits is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.187.890 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.188.016 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.188.127 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.188.409 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.188.734 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.188.830 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.188.918 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.189.131 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.189.348 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.189.445 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.189.558 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.189.784 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.190.005 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPoolGrad is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.190.279 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AssignAdd is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.190.420 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2DBackpropInput is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.190.621 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2DBackpropFilter is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.190.838 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.191.012 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.191.228 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPoolGrad is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.191.394 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2DBackpropFilter is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.191.515 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.191.683 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.191.792 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.191.893 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.191.993 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.192.091 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.192.186 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.192.332 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.192.902 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 153.59 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.193.940 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 983.5 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.194.904 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 921.55 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.196.579 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 1628.53 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.198.106 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_4_insert_type_transform_op in 1476.97 us [WARNING] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.198.742 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:179] GEBackendOptimizeACLAfterKernelSelect] [PROF]ascend_backend_optimize_acl_after_kernel_select costs 6040 usec. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.198.857 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_0_DropoutGenMask in 0.91 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.199.204 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_1_cse in 310.33 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.200.426 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_redundant_op in 1182.39 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.200.613 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_3_eliminate_maketuple_getitem in 147.98 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.200.642 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_4_MergeTransData in 1.43 us [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.201.641 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive GetNext [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:32.201.913 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:467] ConvertAny] Value: ValueTuple [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.202.039 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive GetNext [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.202.068 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.202.187 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.202.211 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive OneHot [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.202.370 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive OneHot [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.202.395 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2D [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:32.202.465 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:32.202.506 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:32.202.541 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.202.623 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2D [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.202.647 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.202.719 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.202.740 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPool [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.202.849 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPool [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.202.872 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2D [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:32.202.925 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:32.202.957 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:32.202.982 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.203.056 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2D [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.203.078 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.203.148 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.203.169 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPool [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.203.250 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPool [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.203.273 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Flatten [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.203.354 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Flatten [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.203.376 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.203.494 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.203.517 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.203.688 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.203.713 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.203.780 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.203.802 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.203.891 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.203.913 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.204.011 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.204.035 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.204.101 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.204.121 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.204.205 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.204.226 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.204.319 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.204.341 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.204.419 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.204.441 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive SoftmaxCrossEntropyWithLogits [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.204.546 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive SoftmaxCrossEntropyWithLogits [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.204.570 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.204.649 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.204.681 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.204.760 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.204.782 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReduceMean [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.204.893 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReduceMean [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.204.916 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.205.015 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.205.038 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.205.128 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.205.152 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.205.312 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.205.419 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.205.442 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.205.501 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.205.579 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.205.600 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReluGrad [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.205.670 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReluGrad [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.205.698 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.205.782 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.205.803 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.205.878 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.205.909 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.205.965 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.206.035 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.206.055 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.206.134 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.206.156 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.206.212 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.206.280 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.206.300 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReluGrad [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.206.370 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReluGrad [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.206.391 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.206.466 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.206.486 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.206.574 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.206.595 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.206.651 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.206.721 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.206.740 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.206.818 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.206.839 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.206.907 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.206.979 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.207.000 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.207.092 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.207.113 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPoolGrad [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.207.237 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPoolGrad [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.207.261 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReluGrad [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.207.338 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReluGrad [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.207.358 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive AssignAdd [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.207.437 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive AssignAdd [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.207.458 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2DBackpropInput [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:32.207.537 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:32.207.566 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.207.646 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2DBackpropInput [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.207.670 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2DBackpropFilter [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:32.207.730 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:32.207.762 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:32.207.787 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.207.865 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2DBackpropFilter [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.207.887 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.207.947 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.208.031 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.208.052 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.208.184 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.208.207 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPoolGrad [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.208.312 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPoolGrad [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.208.337 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReluGrad [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.208.408 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReluGrad [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.208.431 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2DBackpropFilter [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:32.208.492 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:32.208.526 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:37:32.208.551 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.208.635 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2DBackpropFilter [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.208.657 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.208.719 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.208.794 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.208.814 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.208.913 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.208.935 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.209.027 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.209.058 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.209.151 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.209.175 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.209.266 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.209.287 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.209.379 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.209.399 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.209.488 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.209.510 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.209.577 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.209.598 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.209.701 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [WARNING] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.209.726 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:262] CreateKernel] [PROF]create_kernel costs 8126 usec. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.209.977 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/Reshape-op0, index: 0 to input Default/GetNext-op1, index: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.210.011 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1, index: 0 to input Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2, index: 0 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.210.037 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/Reshape-op1, index: 0 to input Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1, index: 0 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.210.062 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/Reshape-op2, index: 0 to input Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/Mul-op0, index: 0 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.210.095 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/gradFlatten-expand/Reshape-op1, index: 0 to input Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op6, index: 0 [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.210.111 [mindspore/ccsrc/backend/common/session/session_basic.cc:1140] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 1 start [INFO] DEBUG(164039,ffff87fd6440,python):2024-01-10-11:37:32.210.125 [mindspore/ccsrc/debug/summary/summary.cc:33] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 1 start [INFO] DEBUG(164039,ffff87fd6440,python):2024-01-10-11:37:32.210.140 [mindspore/ccsrc/debug/summary/summary.cc:54] RecurseSetSummaryNodesForAllGraphs] The total summary nodes is: 0 for graph: 1 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.210.755 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8's input.Send node Default/StreamSend-op0 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op7, recv node Default/StreamRecv-op0 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.210.820 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8's output.Send node Default/StreamSend-op1 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8, recv node Default/StreamRecv-op1 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op8 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.210.874 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9's input.Send node Default/StreamSend-op2 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op3, recv node Default/StreamRecv-op2 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.210.917 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9's output.Send node Default/StreamSend-op3 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9, recv node Default/StreamRecv-op3 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op9 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.210.971 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10's input.Send node Default/StreamSend-op4 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op8, recv node Default/StreamRecv-op4 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.014 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10's output.Send node Default/StreamSend-op5 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10, recv node Default/StreamRecv-op5 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op10 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.074 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11's input.Send node Default/StreamSend-op6 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op4, recv node Default/StreamRecv-op6 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.119 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11's output.Send node Default/StreamSend-op7 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11, recv node Default/StreamRecv-op7 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op11 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.171 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12's input.Send node Default/StreamSend-op8 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op9, recv node Default/StreamRecv-op8 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.214 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12's output.Send node Default/StreamSend-op9 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12, recv node Default/StreamRecv-op9 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op12 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.263 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13's input.Send node Default/StreamSend-op10 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op5, recv node Default/StreamRecv-op10 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.307 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13's output.Send node Default/StreamSend-op11 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13, recv node Default/StreamRecv-op11 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op13 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.369 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14's input.Send node Default/StreamSend-op12 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/gradConv2D-expand/Conv2DBackpropFilter-op1, recv node Default/StreamRecv-op12 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.423 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14's output.Send node Default/StreamSend-op13 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14, recv node Default/StreamRecv-op13 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op14 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.476 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15's input.Send node Default/StreamSend-op14 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv1-Conv2d/gradConv2D-expand/Conv2DBackpropFilter-op1, recv node Default/StreamRecv-op14 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.519 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15's output.Send node Default/StreamSend-op15 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15, recv node Default/StreamRecv-op15 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op15 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.633 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.696 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:0 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.721 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.740 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:0 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.759 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.773 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:1 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.790 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.805 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:2 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.822 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.839 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:2 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.857 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.873 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:3 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.890 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.912 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:1 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.933 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.947 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:4 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.963 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.977 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:4 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.211.994 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.008 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:5 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.025 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.040 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:3 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.059 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.075 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:6 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.093 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.109 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:6 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.128 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.142 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:7 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.159 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.174 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:5 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.194 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.210 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:8 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.226 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.241 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:8 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.258 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.273 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:9 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.289 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.309 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:7 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.326 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.340 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:10 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.357 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.372 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:10 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.388 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.402 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:11 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.417 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.431 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:9 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.452 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.466 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:12 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.483 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.497 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:12 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.513 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.527 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:13 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.544 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.559 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:11 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.579 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.594 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:14 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.611 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.625 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:14 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.643 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.657 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:15 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.674 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.690 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:13 [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.717 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.732 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:15 [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.750 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:891] PrintGraphExecuteOrder] Graph 1 execution order: [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.792 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[0], node name[Default/GetNext-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:outputs{[0]: ValueNode GetNext}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.849 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[1], node name[Default/Reshape-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_Reshape, [1]: CNode_278, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[32])}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.914 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[2], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/OneHot-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_OneHot, [1]: 319, [2]: ValueNode Tensor(shape=[], dtype=Int64, value=10), [3]: ValueNode Tensor(shape=[], dtype=Float32, value=1), [4]: ValueNode Tensor(shape=[], dtype=Float32, value=0), [5]: ValueNode -1}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.212.967 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[3], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv1-Conv2d/Conv2D-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode Conv2D, [1]: CNode_278, [2]: equiv_CNode_320}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.015 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[4], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op4], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_ReLU, [1]: equiv_CNode_278}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.062 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[5], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op3], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode MaxPool, [1]: equiv_CNode_278}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.115 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[6], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/Conv2D-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode Conv2D, [1]: equiv_CNode_278, [2]: equiv_CNode_321}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.161 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[7], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op5], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_ReLU, [1]: equiv_CNode_278}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.212 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[8], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode MaxPool, [1]: equiv_CNode_278}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.266 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[9], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_Flatten, [1]: equiv_CNode_278}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.326 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[10], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/MatMul-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode MatMul, [1]: equiv_CNode_278, [2]: equiv_CNode_322}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.381 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[11], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/BiasAdd-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_BiasAdd, [1]: equiv_CNode_278, [2]: equiv_CNode_323, [3]: ValueNode 0}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.426 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[12], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op6], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_ReLU, [1]: equiv_CNode_278}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.475 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[13], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/MatMul-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode MatMul, [1]: equiv_CNode_278, [2]: equiv_CNode_324}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.530 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[14], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/BiasAdd-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_BiasAdd, [1]: equiv_CNode_278, [2]: equiv_CNode_325, [3]: ValueNode 0}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.575 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[15], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op7], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_ReLU, [1]: equiv_CNode_278}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.625 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[16], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/MatMul-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode MatMul, [1]: equiv_CNode_278, [2]: equiv_CNode_326}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.674 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[17], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_BiasAdd, [1]: equiv_CNode_278, [2]: equiv_CNode_317, [3]: ValueNode 0}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.749 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[18], node name[Default/Reshape-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_Reshape, [1]: equiv_CNode_278, [2]: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10])}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.790 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[19], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/SoftmaxCrossEntropyWithLogits-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode SoftmaxCrossEntropyWithLogits, [1]: 319, [2]: 319}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.825 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[20], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/Mul-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_Mul, [1]: 319, [2]: ValueNode Tensor(shape=[32, 1], dtype=Float32, value=[...])}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.862 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[21], node name[Default/Reshape-op2], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_Reshape, [1]: 319, [2]: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10])}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.905 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[22], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/ReduceMean-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_ReduceMean, [1]: 319, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), [3]: ValueNode false}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.943 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[23], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op10], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 319, [2]: equiv_CNode_326}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.213.986 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[24], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op7], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 319, [2]: equiv_CNode_278}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.018 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[25], node name[Default/StreamSend-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_328{[0]: ValueNode StreamSend}], event id[0] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.044 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[26], node name[Default/StreamRecv-op0], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_329{[0]: ValueNode StreamRecv}], event id[0] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.096 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[27], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 327}], group[hccl_world_group] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.122 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[28], node name[Default/StreamSend-op1], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_330{[0]: ValueNode StreamSend}], event id[1] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.155 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[29], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op3], logic id[4294967295], stream id[0], node info[@kernel_graph_1:331{[0]: ValueNode PrimFunc_BiasAddGrad, [1]: 319, [2]: ValueNode 0}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.182 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[30], node name[Default/StreamSend-op2], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_332{[0]: ValueNode StreamSend}], event id[2] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.207 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[31], node name[Default/StreamRecv-op2], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_333{[0]: ValueNode StreamRecv}], event id[2] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.246 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[32], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 331}], group[hccl_world_group] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.271 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[33], node name[Default/StreamSend-op3], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_334{[0]: ValueNode StreamSend}], event id[3] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.295 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[34], node name[Default/StreamRecv-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_335{[0]: ValueNode StreamRecv}], event id[1] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.351 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[35], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op8], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.398 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[36], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/gradReLU-expand/ReluGrad-op4], logic id[4294967295], stream id[0], node info[@kernel_graph_1:336{[0]: ValueNode PrimFunc_ReluGrad, [1]: 327, [2]: equiv_CNode_278}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.437 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[37], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op11], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 336, [2]: equiv_CNode_324}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.484 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[38], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op8], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 336, [2]: equiv_CNode_278}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.510 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[39], node name[Default/StreamSend-op4], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_337{[0]: ValueNode StreamSend}], event id[4] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.535 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[40], node name[Default/StreamRecv-op4], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_338{[0]: ValueNode StreamRecv}], event id[4] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.574 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[41], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 327}], group[hccl_world_group] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.600 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[42], node name[Default/StreamSend-op5], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_339{[0]: ValueNode StreamSend}], event id[5] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.625 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[43], node name[Default/StreamRecv-op3], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_340{[0]: ValueNode StreamRecv}], event id[3] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.676 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[44], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op9], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.710 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[45], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op4], logic id[4294967295], stream id[0], node info[@kernel_graph_1:331{[0]: ValueNode PrimFunc_BiasAddGrad, [1]: 336, [2]: ValueNode 0}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.737 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[46], node name[Default/StreamSend-op6], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_341{[0]: ValueNode StreamSend}], event id[6] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.763 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[47], node name[Default/StreamRecv-op6], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_342{[0]: ValueNode StreamRecv}], event id[6] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.802 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[48], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 331}], group[hccl_world_group] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.835 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[49], node name[Default/StreamSend-op7], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_343{[0]: ValueNode StreamSend}], event id[7] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.859 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[50], node name[Default/StreamRecv-op5], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_344{[0]: ValueNode StreamRecv}], event id[5] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.912 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[51], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op10], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.956 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[52], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/gradReLU-expand/ReluGrad-op5], logic id[4294967295], stream id[0], node info[@kernel_graph_1:336{[0]: ValueNode PrimFunc_ReluGrad, [1]: 327, [2]: equiv_CNode_278}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.214.990 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[53], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op6], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 336, [2]: equiv_CNode_322}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.039 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[54], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op9], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 336, [2]: equiv_CNode_278}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.065 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[55], node name[Default/StreamSend-op8], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_345{[0]: ValueNode StreamSend}], event id[8] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.089 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[56], node name[Default/StreamRecv-op8], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_346{[0]: ValueNode StreamRecv}], event id[8] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.131 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[57], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 327}], group[hccl_world_group] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.155 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[58], node name[Default/StreamSend-op9], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_347{[0]: ValueNode StreamSend}], event id[9] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.178 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[59], node name[Default/StreamRecv-op7], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_348{[0]: ValueNode StreamRecv}], event id[7] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.238 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[60], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op11], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.270 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[61], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op5], logic id[4294967295], stream id[0], node info[@kernel_graph_1:331{[0]: ValueNode PrimFunc_BiasAddGrad, [1]: 336, [2]: ValueNode 0}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.293 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[62], node name[Default/StreamSend-op10], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_349{[0]: ValueNode StreamSend}], event id[10] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.316 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[63], node name[Default/StreamRecv-op10], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_350{[0]: ValueNode StreamRecv}], event id[10] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.353 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[64], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 331}], group[hccl_world_group] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.376 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[65], node name[Default/StreamSend-op11], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_351{[0]: ValueNode StreamSend}], event id[11] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.399 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[66], node name[Default/StreamRecv-op9], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_352{[0]: ValueNode StreamRecv}], event id[9] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.448 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[67], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op12], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.486 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[68], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/gradFlatten-expand/Reshape-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:353{[0]: ValueNode PrimFunc_Reshape, [1]: 327, [2]: ValueNode (32, 16, 5, 5)}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.545 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[69], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/gradMaxPool-expand/MaxPoolGrad-op2], logic id[4294967295], stream id[0], node info[@kernel_graph_1:354{[0]: ValueNode MaxPoolGrad, [1]: equiv_CNode_278, [2]: equiv_CNode_278, [3]: 353}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.597 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[70], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/gradReLU-expand/ReluGrad-op6], logic id[4294967295], stream id[0], node info[@kernel_graph_1:336{[0]: ValueNode PrimFunc_ReluGrad, [1]: 354, [2]: equiv_CNode_278}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.652 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[71], node name[Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AssignAdd, [1]: param_global_step, [2]: ValueNode Tensor(shape=[1], dtype=Int32, value=[1]), [3]: CNode_355}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.694 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[72], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/gradConv2D-expand/Conv2DBackpropInput-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:356{[0]: ValueNode Conv2DBackpropInput, [1]: 336, [2]: equiv_CNode_321, [3]: ValueNode (32, 6, 14, 14)}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.741 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[73], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/gradConv2D-expand/Conv2DBackpropFilter-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:356{[0]: ValueNode Conv2DBackpropFilter, [1]: 336, [2]: equiv_CNode_278, [3]: ValueNode (16, 6, 5, 5)}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.766 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[74], node name[Default/StreamSend-op12], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_357{[0]: ValueNode StreamSend}], event id[12] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.791 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[75], node name[Default/StreamRecv-op12], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_358{[0]: ValueNode StreamRecv}], event id[12] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.830 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[76], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 356}], group[hccl_world_group] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.855 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[77], node name[Default/StreamSend-op13], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_359{[0]: ValueNode StreamSend}], event id[13] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.881 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[78], node name[Default/StreamRecv-op11], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_360{[0]: ValueNode StreamRecv}], event id[11] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.215.933 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[79], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op13], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.216.014 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[80], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc3.bias, [2]: param_moments.fc3.bias, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_361}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.216.072 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[81], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/gradMaxPool-expand/MaxPoolGrad-op3], logic id[4294967295], stream id[0], node info[@kernel_graph_1:354{[0]: ValueNode MaxPoolGrad, [1]: equiv_CNode_278, [2]: equiv_CNode_278, [3]: 356}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.216.115 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[82], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/gradReLU-expand/ReluGrad-op7], logic id[4294967295], stream id[0], node info[@kernel_graph_1:336{[0]: ValueNode PrimFunc_ReluGrad, [1]: 354, [2]: equiv_CNode_278}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.216.155 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[83], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv1-Conv2d/gradConv2D-expand/Conv2DBackpropFilter-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:362{[0]: ValueNode Conv2DBackpropFilter, [1]: 336, [2]: CNode_278, [3]: ValueNode (6, 1, 5, 5)}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.216.181 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[84], node name[Default/StreamSend-op14], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_363{[0]: ValueNode StreamSend}], event id[14] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.216.205 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[85], node name[Default/StreamRecv-op14], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_364{[0]: ValueNode StreamRecv}], event id[14] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.216.243 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[86], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 362}], group[hccl_world_group] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.216.269 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[87], node name[Default/StreamSend-op15], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_365{[0]: ValueNode StreamSend}], event id[15] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.216.294 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[88], node name[Default/StreamRecv-op13], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_366{[0]: ValueNode StreamRecv}], event id[13] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.216.345 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[89], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op14], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.216.425 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[90], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc3.weight, [2]: param_moments.fc3.weight, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_367}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.216.495 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[91], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc2.bias, [2]: param_moments.fc2.bias, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_368}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.216.562 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[92], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc2.weight, [2]: param_moments.fc2.weight, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_369}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.216.631 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[93], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc1.bias, [2]: param_moments.fc1.bias, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_370}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.216.701 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[94], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc1.weight, [2]: param_moments.fc1.weight, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_371}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.216.768 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[95], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_conv2.weight, [2]: param_moments.conv2.weight, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_372}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.216.792 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[96], node name[Default/StreamRecv-op15], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_373{[0]: ValueNode StreamRecv}], event id[15] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.216.843 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[97], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op15], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.216.909 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[98], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_conv1.weight, [2]: param_moments.conv1.weight, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_374}] [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.217.406 [mindspore/ccsrc/backend/common/somas/somas.cc:143] Assign] Start Somas Assign for graph 1 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.217.853 [mindspore/ccsrc/backend/common/somas/somas.cc:151] Assign] Somas Configure success, configuration info: Device Name: Ascend Run by execution order: 1 Enable debug log: 0 Debug log path: . [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.217.875 [mindspore/ccsrc/backend/common/somas/somas.cc:154] Assign] Start Initialize SOMAS Model [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.219.244 [mindspore/ccsrc/backend/common/somas/somas.cc:1065] GraphOutputProcess] Set 1 graph output tensors' aligned size to 0. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.219.883 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 1 output 2 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.219.921 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 9 output 10 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.219.945 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 18 output 19 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.219.961 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 22 output 23 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.220.005 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 40 output 47 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.220.042 [mindspore/ccsrc/backend/common/somas/somas.cc:1135] RefNodeProcess] Special Tensor total size: RefNode: input 107008 output 351828 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.220.109 [mindspore/ccsrc/backend/common/somas/somas.cc:555] InitSomasModel] Created 2 streams (0 groups), 99 nodes, 69 tensors, 5 union tensors lists, and 0 contiguous tensors lists [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.220.712 [mindspore/ccsrc/backend/common/somas/somas.cc:157] Assign] End Initialize SOMAS Model [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.220.728 [mindspore/ccsrc/backend/common/somas/somas.cc:176] Assign] Start Computing Conflict Matrix [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.220.741 [mindspore/ccsrc/backend/common/somas/somas.cc:1286] ComputeBasicMatrix] Start Conflict Computing (Bitset Model) [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.220.761 [mindspore/ccsrc/backend/common/somas/somas.cc:1291] ComputeBasicMatrix] Start Bitset [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.220.800 [mindspore/ccsrc/backend/common/somas/somas.cc:1299] ComputeBasicMatrix] Start Path Computing [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.220.827 [mindspore/ccsrc/backend/common/somas/somas.cc:1307] ComputeBasicMatrix] End Path Computing [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.220.837 [mindspore/ccsrc/backend/common/somas/somas.cc:1309] ComputeBasicMatrix] Start Tensor Relation Computing [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.220.906 [mindspore/ccsrc/backend/common/somas/somas.cc:1462] ComputeMultiTensorConflicts] Start Computing Conflicts Pairs, tensors list size is 60 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.221.009 [mindspore/ccsrc/backend/common/somas/somas.cc:1469] ComputeMultiTensorConflicts] End Computing Conflicts Pairs (time taken 0ms) [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.221.021 [mindspore/ccsrc/backend/common/somas/somas.cc:1367] ComputeBasicMatrix] End Basic Conflict Computing (Bitset Model)(time taken 0ms) [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.221.062 [mindspore/ccsrc/backend/common/somas/somas.cc:178] Assign] End Computing Conflict Matrix [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.221.075 [mindspore/ccsrc/backend/common/somas/somas.cc:1533] Solve] Somas Assign start... [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.221.107 [mindspore/ccsrc/backend/common/somas/somas.cc:1555] Solve] Start Solving [INFO] PRE_ACT(164039,fffe9dffb0f0,python):2024-01-10-11:37:32.221.339 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 55 tensors [INFO] PRE_ACT(164039,fffe9e7fc0f0,python):2024-01-10-11:37:32.221.358 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 55 tensors [INFO] PRE_ACT(164039,fffe9cff90f0,python):2024-01-10-11:37:32.221.356 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 55 tensors [INFO] PRE_ACT(164039,fffe9d7fa0f0,python):2024-01-10-11:37:32.221.368 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 55 tensors [INFO] PRE_ACT(164039,fffe9dffb0f0,python):2024-01-10-11:37:32.221.498 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 2/4 2196992 Bytes (0.00204611 GB) Shared Objects size(>), index(<) smallest [INFO] PRE_ACT(164039,fffe9e7fc0f0,python):2024-01-10-11:37:32.221.528 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 3/4 2196992 Bytes (0.00204611 GB) Single Object size(>), index(<) bestfit [INFO] PRE_ACT(164039,fffe9cff90f0,python):2024-01-10-11:37:32.221.535 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 1/4 2196992 Bytes (0.00204611 GB) Shared Objects size(>), index(<) bestfit [INFO] PRE_ACT(164039,fffe9d7fa0f0,python):2024-01-10-11:37:32.221.569 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 4/4 2196992 Bytes (0.00204611 GB) Single Object size(>), index(<) smallest [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.221.597 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:176] Solving] SOMAS SOLVER RESUME: [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.221.612 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:177] Solving] Best Solution:[1/4] [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.221.631 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:178] Solving] Best result:2196992 Bytes 0.00204611 GB (0.00204611 GB + 0 GB from lifelong tensors) [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.221.645 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:181] Solving] Best timing:0 ms [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.221.656 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:182] Solving] Best algorithm: Shared Objects [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.221.666 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:183] Solving] Best sorting strategy: size(>), index(<) [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.221.677 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:184] Solving] Best offset strategy: bestfit [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.221.695 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:185] Solving] Time elapsed: 0 ms [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.221.709 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:186] Solving] Spread:0 %% [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.221.773 [mindspore/ccsrc/backend/common/somas/somas.cc:1564] Solve] End Solving [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.221.848 [mindspore/ccsrc/backend/common/somas/somas.cc:2096] GenGraphStatisticInfo] Lower Bound: 2186752 (0.00203657 GB), Upper Bound: 4660224 (0.00434017 GB) [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.221.869 [mindspore/ccsrc/backend/common/somas/somas.cc:2099] GenGraphStatisticInfo] Total Dynamic Size (Upper Bound): 4660224 Theoretical Optimal Size (Lower Bound): 2186752 Total Workspace Size: 0 Total Communication Input Tensor Size: 248832 Total Communication Output Tensor Size: 0 Total LifeLong All Tensor Size: 0 Total LifeLong Start Tensor Size: 0 Total LifeLong End Tensor Size: 512 Reused Size(Allocate Size): 0 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.221.882 [mindspore/ccsrc/backend/common/somas/somas.cc:1583] Solve] Somas Assign end. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.222.023 [mindspore/ccsrc/backend/common/somas/somas.cc:380] UpdateSomasResultToGraph] Merged Block size: 12 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.222.039 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 0, offset: 1205248, size: 602624 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.222.050 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 1, offset: 602624, size: 602624 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.222.061 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 2, offset: 0, size: 602624 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.222.072 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 3, offset: 1807872, size: 192512 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.222.082 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 4, offset: 2000384, size: 131584 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.222.093 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 5, offset: 2131968, size: 40448 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.222.103 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 6, offset: 2182144, size: 9728 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.222.113 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 7, offset: 2172416, size: 9728 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.222.125 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 8, offset: 2191872, size: 3584 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.222.138 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 9, offset: 2196480, size: 512 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:37:32.222.150 [mindspore/ccsrc/backend/common/somas/somas.cc:189] Assign] Somas Allocate end. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.222.161 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:310] DoSomas] Somas allocate success for graph 1 somas size: 2196992 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.222.376 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:801] CreateDeviceAddress] Status record: start create device address. graph id: 1 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.223.298 [mindspore/ccsrc/runtime/device/device_address_utils.cc:454] CreateValueNodeDeviceAddress] No device address for value node:Default/data-17, debug name:ValueNode U [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.225.643 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is conv2.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.225.683 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is conv2.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.225.727 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc2.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.225.747 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc2.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.225.765 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc1.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.225.782 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc1.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.225.798 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc3.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.225.816 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc3.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.225.831 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/GetNext-op1, index is 1; cur kernel is Default/Reshape-op0, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.225.864 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/GetNext-op1, index is 1; cur kernel is Default/Reshape-op0, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.225.887 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc3.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.225.905 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc3.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.225.921 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/Mul-op0, index is 0; cur kernel is Default/Reshape-op2, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.225.945 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/Mul-op0, index is 0; cur kernel is Default/Reshape-op2, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.225.965 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc1.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.225.989 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc1.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.226.006 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc2.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.226.023 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc2.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.226.039 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is conv1.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.226.056 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is conv1.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.226.075 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is global_step, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.226.091 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is global_step, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.226.106 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1, index is 0; cur kernel is Default/Reshape-op1, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.226.130 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1, index is 0; cur kernel is Default/Reshape-op1, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.226.151 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2, index is 0; cur kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.226.174 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2, index is 0; cur kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.226.192 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op6, index is 0; cur kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/gradFlatten-expand/Reshape-op1, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:37:32.226.223 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op6, index is 0; cur kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/gradFlatten-expand/Reshape-op1, index is 0 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.226.242 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:808] CreateDeviceAddress] Status record: end create device address. graph id: 1 [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.226.431 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1129] CacheGraphOutputToFrontNodeWithIndex] Get graph backend output nodes. [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.226.468 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1137] CacheGraphOutputToFrontNodeWithIndex] Get graph front output nodes. [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:37:32.226.495 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1155] CacheGraphOutputToFrontNodeWithIndex] Backend output: Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/ReduceMean-op0 with index: 0 map to front node: Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits-op0 with index: 0 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.226.603 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:507] CompileGraph] Status record: end compile graph. graph id: 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:37:32.227.056 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:805] CompileGraphFromSegment] Compile cut segment, the cut node: @381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316:CNode_375{[0]: ValueNode Return, [1]: CNode_278} [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.227.244 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:521] Transform] Graph(kernel_graph_1) transforms actor begin, strategy:pipeline_with_execution_order [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.227.316 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2619] PersistDeviceTensorForValueNode] The device address is not exist: ValueNode_376(U) [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.227.503 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1101] BuildDataSourceActor] Create queue data source actor: kernel_graph_1_DeviceDSActor_1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.228.445 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1232] BuildLoopCountActor] Create loop count actor: kernel_graph_1_LoopCountActor [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.228.482 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1265] BuildOutputActor] Create output actor: kernel_graph_1_OutputActor [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.228.501 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1285] BuildDataPrepareActor] Create data prepare actor: kernel_graph_1_DataPrepareActor [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.228.638 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:866] CacheGraphOutputToActor] Cache graph 1 output node:Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/ReduceMean-op0 debug string:@kernel_graph_1:319{[0]: ValueNode PrimFunc_ReduceMean, [1]: 319, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), [3]: ValueNode false} with index:0 to actor:Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/ReduceMean-op0, from front node:Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits-op0 debug string:@381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316:equiv_CNode_278{[0]: ValueNode SparseSoftmaxCrossEntropyWithLogits, [1]: equiv_CNode_278, [2]: CNode_278} with index:0 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.228.675 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:809] AddSomasInfoForGraphOutput] The graph 1 output node:Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/ReduceMean-op0 with index: 0 somas enable or not: 1, somas offset: 2195456, aligned size: 512 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.228.700 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1518] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_1 start. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.228.854 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1520] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_1 end. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.229.962 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1 add input node monad device tensor store:global_step [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.229.995 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1 add input node monad device tensor store:Default/network-TrainOneStepCell/optimizer-Momentum/data-0 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.153 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.187 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.211 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 add input node monad device tensor store:fc3.bias [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.227 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 add input node monad device tensor store:moments.fc3.bias [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.240 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.259 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.445 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.472 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/MatMul-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.495 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op10, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.515 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 add input node monad device tensor store:fc3.weight [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.529 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 add input node monad device tensor store:moments.fc3.weight [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.540 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.551 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.606 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.630 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/BiasAdd-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.656 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 add input node monad device tensor store:fc2.bias [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.669 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 add input node monad device tensor store:moments.fc2.bias [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.680 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.691 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.743 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.767 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/MatMul-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.788 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op11, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.805 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 add input node monad device tensor store:fc2.weight [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.817 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 add input node monad device tensor store:moments.fc2.weight [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.827 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.838 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.895 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.919 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/BiasAdd-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.939 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 add input node monad device tensor store:fc1.bias [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.954 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 add input node monad device tensor store:moments.fc1.bias [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.965 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.230.976 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.027 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.053 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/MatMul-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.078 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op6, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.095 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 add input node monad device tensor store:fc1.weight [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.108 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 add input node monad device tensor store:moments.fc1.weight [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.129 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.140 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.197 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.223 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/Conv2D-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.247 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/gradConv2D-expand/Conv2DBackpropInput-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.268 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 add input node monad device tensor store:conv2.weight [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.279 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 add input node monad device tensor store:moments.conv2.weight [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.290 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.300 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.373 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.397 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv1-Conv2d/Conv2D-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.423 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 add input node monad device tensor store:conv1.weight [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.436 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 add input node monad device tensor store:moments.conv1.weight [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.447 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.458 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.471 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.495 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.517 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.539 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.559 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.580 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.601 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.231.620 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.232.456 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 1_actor_set_kernel_graph_1_memory_actor_insert in 24.54 us [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.232.494 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 2_actor_set_kernel_graph_1_invalid_data_arrow_elimination in 6.85 us [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.232.756 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 3_actor_set_kernel_graph_1_multi_actor_fusion in 237.08 us [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.232.806 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 4_actor_set_kernel_graph_1_batch_data_arrow_fusion in 23.03 us [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:37:32.232.827 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:554] Transform] Graph(kernel_graph_1) transforms actor end. [WARNING] VM(164039,ffff87fd6440,python):2024-01-10-11:37:32.234.320 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:612] CompileGraphs] [PROF]compile_func_graph costs 91945 usec. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:37:32.234.366 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:615] CompileGraphs] Status record: end compile function graph: 381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316, produce actor: kernel_graph_1 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.234.390 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end task_emit action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.234.416 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:268] SetLoopCount] Change vm_loop_flag to 0, set loop_size to 468 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.234.434 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start execute action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.234.454 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end execute action. TotalTime = 0.746507, [19] [parse]: 0.0109727 [symbol_resolve]: 0.239377, [1] [Cycle 1]: 0.238719, [1] [resolve]: 0.238691 [graph_reusing]: 7.211e-05 [meta_unpack_prepare]: 0.00109773 [pre_cconv]: 3.111e-05 [abstract_specialize]: 0.233213 [pack_expand]: 0.00045335 [auto_monad]: 0.00321261 [inline]: 3.132e-05 [pre_auto_parallel]: 3.982e-05 [pipeline_split]: 3.531e-05 [optimize]: 0.162838, [35] [py_interpret_to_execute]: 0.00112221 [rewriter_before_opt_a]: 0.00514828 [opt_a]: 0.149624, [3] [Cycle 1]: 0.119715, [30] [expand_dump_flag]: 6.097e-05 [switch_simplify]: 0.00099878 [a_1]: 0.0167006 [recompute_prepare]: 0.00014378 [updatestate_depend_eliminate]: 0.00077842 [updatestate_assign_eliminate]: 0.0002023 [updatestate_loads_eliminate]: 0.0004623 [parameter_eliminate]: 1.934e-05 [a_2]: 0.00272656 [accelerated_algorithm]: 8.531e-05 [pynative_shard]: 4.623e-05 [auto_parallel]: 4.82e-06 [parallel]: 3.541e-05 [merge_comm]: 7.805e-05 [allreduce_fusion]: 3.723e-05 [virtual_dataset]: 8.209e-05 [get_grad_eliminate_]: 6.486e-05 [virtual_output]: 6.223e-05 [merge_forward]: 0.00011243 [cell_reuse_recompute_pass]: 8.59996e-07 [cell_reuse_handle_not_recompute_node_pass]: 0.00017322 [meta_fg_expand]: 0.0326599 [after_resolve]: 0.00038512 [a_after_grad]: 0.00059045 [renormalize]: 0.0530532 [real_op_eliminate]: 0.00063154 [auto_monad_grad]: 0.00047222 [auto_monad_eliminator]: 0.00081017 [cse]: 0.0024231 [a_3]: 0.00548508 [Cycle 2]: 0.021074, [30] [expand_dump_flag]: 2.266e-05 [switch_simplify]: 0.00029626 [a_1]: 0.00843789 [recompute_prepare]: 5.548e-05 [updatestate_depend_eliminate]: 0.00052712 [updatestate_assign_eliminate]: 7.751e-05 [updatestate_loads_eliminate]: 0.00015751 [parameter_eliminate]: 3.76999e-06 [a_2]: 0.00091489 [accelerated_algorithm]: 6.831e-05 [pynative_shard]: 6.183e-05 [auto_parallel]: 6.14e-06 [parallel]: 9.95e-06 [merge_comm]: 4.183e-05 [allreduce_fusion]: 2.543e-05 [virtual_dataset]: 5.208e-05 [get_grad_eliminate_]: 4.429e-05 [virtual_output]: 4.335e-05 [merge_forward]: 7.121e-05 [cell_reuse_recompute_pass]: 7.29997e-07 [cell_reuse_handle_not_recompute_node_pass]: 0.00011362 [meta_fg_expand]: 8.686e-05 [after_resolve]: 5.862e-05 [a_after_grad]: 5.598e-05 [renormalize]: 0.008043 [real_op_eliminate]: 6.89e-05 [auto_monad_grad]: 4.39e-06 [auto_monad_eliminator]: 0.00026591 [cse]: 0.00083931 [a_3]: 0.0003973 [Cycle 3]: 0.00365612, [30] [expand_dump_flag]: 1.47e-06 [switch_simplify]: 4.93e-05 [a_1]: 0.00071412 [recompute_prepare]: 4.399e-05 [updatestate_depend_eliminate]: 0.00010776 [updatestate_assign_eliminate]: 6.68e-05 [updatestate_loads_eliminate]: 6.88e-05 [parameter_eliminate]: 2.27999e-06 [a_2]: 0.00091371 [accelerated_algorithm]: 7.01e-05 [pynative_shard]: 4.393e-05 [auto_parallel]: 4.66999e-06 [parallel]: 7.18e-06 [merge_comm]: 2.929e-05 [allreduce_fusion]: 1.883e-05 [virtual_dataset]: 5.285e-05 [get_grad_eliminate_]: 4.609e-05 [virtual_output]: 4.529e-05 [merge_forward]: 6.509e-05 [cell_reuse_recompute_pass]: 4.79995e-07 [cell_reuse_handle_not_recompute_node_pass]: 0.000119 [meta_fg_expand]: 5.905e-05 [after_resolve]: 5.915e-05 [a_after_grad]: 5.73e-05 [renormalize]: 5.99975e-08 [real_op_eliminate]: 4.62e-05 [auto_monad_grad]: 2.72e-06 [auto_monad_eliminator]: 0.00014952 [cse]: 0.00026417 [a_3]: 0.00038665 [py_interpret_to_execute_after_opt_a]: 8.98e-05 [slice_cell_reuse_recomputed_activation]: 3.15e-06 [rewriter_after_opt_a]: 0.00190757 [convert_after_rewriter]: 8.857e-05 [order_py_execute_after_rewriter]: 5.703e-05 [opt_b]: 0.00182141, [1] [Cycle 1]: 0.00181482, [7] [b_1]: 0.00121472 [b_2]: 5.39e-05 [updatestate_depend_eliminate]: 6.791e-05 [updatestate_assign_eliminate]: 6.381e-05 [updatestate_loads_eliminate]: 6.938e-05 [renormalize]: 3.906e-05 [cse]: 0.00025951 [cconv]: 6.262e-05 [opt_after_cconv]: 0.00069915, [1] [Cycle 1]: 0.00069224, [7] [c_1]: 0.00016665 [parameter_eliminate]: 2.11001e-06 [updatestate_depend_eliminate]: 7.605e-05 [updatestate_assign_eliminate]: 6.584e-05 [updatestate_loads_eliminate]: 6.679e-05 [cse]: 0.00023881 [renormalize]: 3.628e-05 [remove_dup_value]: 0.0002906 [tuple_transform]: 0.00047042, [1] [Cycle 1]: 0.00046414, [3] [d_1]: 0.00028418 [d_2]: 0.00012252 [renormalize]: 3.634e-05 [add_cache_embedding]: 0.00013574 [add_recomputation]: 0.00060249 [cse_after_recomputation]: 0.00022044, [1] [Cycle 1]: 0.00021402, [1] [cse]: 0.00020491 [environ_conv]: 8.051e-05 [label_micro_interleaved_index]: 3.01e-06 [label_fine_grained_interleaved_index]: 2.59e-06 [assign_add_opt]: 3.697e-05 [slice_recompute_activation]: 2.76e-06 [micro_interleaved_order_control]: 2.6e-06 [full_micro_interleaved_order_control]: 1.95e-06 [comp_comm_scheduling]: 2.42001e-06 [reorder_send_recv_between_fp_bp]: 2.14999e-06 [comm_op_add_attrs]: 1.06e-06 [add_comm_op_reuse_tag]: 1.861e-05 [overlap_opt_shard_in_pipeline]: 1.702e-05 [grouped_pairwise_exchange_alltoall]: 1.754e-05 [overlap_recompute_and_grad_model_parallel]: 1.9e-06 [overlap_grad_matmul_and_grad_allreduce]: 1.214e-05 [split_matmul_comm_elemetwise]: 3.15e-05 [split_layernorm_comm]: 1.94e-06 [process_send_recv_for_ge]: 2.74e-06 [handle_group_info]: 8.30005e-07 [auto_monad_reorder]: 0.00036306 [get_jit_bprop_graph]: 2.875e-05 [eliminate_special_op_node]: 0.00099236 [validate]: 0.00029889 [distribtued_split]: 0.00045429 [task_emit]: 0.092602 [execute]: 3.128e-05 Sums parse : 0.010973s : 1.48% symbol_resolve.resolve : 0.238691s : 32.29% graph_reusing : 0.000072s : 0.01% meta_unpack_prepare : 0.001098s : 0.15% pre_cconv : 0.000031s : 0.00% abstract_specialize : 0.233213s : 31.55% pack_expand : 0.000453s : 0.06% auto_monad : 0.003213s : 0.43% inline : 0.000031s : 0.00% pre_auto_parallel : 0.000040s : 0.01% pipeline_split : 0.000035s : 0.00% optimize.py_interpret_to_execute : 0.001122s : 0.15% optimize.rewriter_before_opt_a : 0.005148s : 0.70% optimize.opt_a.expand_dump_flag : 0.000085s : 0.01% optimize.opt_a.switch_simplify : 0.001344s : 0.18% optimize.opt_a.a_1 : 0.025853s : 3.50% optimize.opt_a.recompute_prepare : 0.000243s : 0.03% optimize.opt_a.updatestate_depend_eliminate : 0.001413s : 0.19% optimize.opt_a.updatestate_assign_eliminate : 0.000347s : 0.05% optimize.opt_a.updatestate_loads_eliminate : 0.000689s : 0.09% optimize.opt_a.parameter_eliminate : 0.000025s : 0.00% optimize.opt_a.a_2 : 0.004555s : 0.62% optimize.opt_a.accelerated_algorithm : 0.000224s : 0.03% optimize.opt_a.pynative_shard : 0.000152s : 0.02% optimize.opt_a.auto_parallel : 0.000016s : 0.00% optimize.opt_a.parallel : 0.000053s : 0.01% optimize.opt_a.merge_comm : 0.000149s : 0.02% optimize.opt_a.allreduce_fusion : 0.000081s : 0.01% optimize.opt_a.virtual_dataset : 0.000187s : 0.03% optimize.opt_a.get_grad_eliminate_ : 0.000155s : 0.02% optimize.opt_a.virtual_output : 0.000151s : 0.02% optimize.opt_a.merge_forward : 0.000249s : 0.03% optimize.opt_a.cell_reuse_recompute_pass : 0.000002s : 0.00% optimize.opt_a.cell_reuse_handle_not_recompute_node_pass : 0.000406s : 0.05% optimize.opt_a.meta_fg_expand : 0.032806s : 4.44% optimize.opt_a.after_resolve : 0.000503s : 0.07% optimize.opt_a.a_after_grad : 0.000704s : 0.10% optimize.opt_a.renormalize : 0.061096s : 8.27% optimize.opt_a.real_op_eliminate : 0.000747s : 0.10% optimize.opt_a.auto_monad_grad : 0.000479s : 0.06% optimize.opt_a.auto_monad_eliminator : 0.001226s : 0.17% optimize.opt_a.cse : 0.003527s : 0.48% optimize.opt_a.a_3 : 0.006269s : 0.85% optimize.py_interpret_to_execute_after_opt_a : 0.000090s : 0.01% optimize.slice_cell_reuse_recomputed_activation : 0.000003s : 0.00% optimize.rewriter_after_opt_a : 0.001908s : 0.26% optimize.convert_after_rewriter : 0.000089s : 0.01% optimize.order_py_execute_after_rewriter : 0.000057s : 0.01% optimize.opt_b.b_1 : 0.001215s : 0.16% optimize.opt_b.b_2 : 0.000054s : 0.01% optimize.opt_b.updatestate_depend_eliminate : 0.000068s : 0.01% optimize.opt_b.updatestate_assign_eliminate : 0.000064s : 0.01% optimize.opt_b.updatestate_loads_eliminate : 0.000069s : 0.01% optimize.opt_b.renormalize : 0.000039s : 0.01% optimize.opt_b.cse : 0.000260s : 0.04% optimize.cconv : 0.000063s : 0.01% optimize.opt_after_cconv.c_1 : 0.000167s : 0.02% optimize.opt_after_cconv.parameter_eliminate : 0.000002s : 0.00% optimize.opt_after_cconv.updatestate_depend_eliminate : 0.000076s : 0.01% optimize.opt_after_cconv.updatestate_assign_eliminate : 0.000066s : 0.01% optimize.opt_after_cconv.updatestate_loads_eliminate : 0.000067s : 0.01% optimize.opt_after_cconv.cse : 0.000239s : 0.03% optimize.opt_after_cconv.renormalize : 0.000036s : 0.00% optimize.remove_dup_value : 0.000291s : 0.04% optimize.tuple_transform.d_1 : 0.000284s : 0.04% optimize.tuple_transform.d_2 : 0.000123s : 0.02% optimize.tuple_transform.renormalize : 0.000036s : 0.00% optimize.add_cache_embedding : 0.000136s : 0.02% optimize.add_recomputation : 0.000602s : 0.08% optimize.cse_after_recomputation.cse : 0.000205s : 0.03% optimize.environ_conv : 0.000081s : 0.01% optimize.label_micro_interleaved_index : 0.000003s : 0.00% optimize.label_fine_grained_interleaved_index : 0.000003s : 0.00% optimize.assign_add_opt : 0.000037s : 0.01% optimize.slice_recompute_activation : 0.000003s : 0.00% optimize.micro_interleaved_order_control : 0.000003s : 0.00% optimize.full_micro_interleaved_order_control : 0.000002s : 0.00% optimize.comp_comm_scheduling : 0.000002s : 0.00% optimize.reorder_send_recv_between_fp_bp : 0.000002s : 0.00% optimize.comm_op_add_attrs : 0.000001s : 0.00% optimize.add_comm_op_reuse_tag : 0.000019s : 0.00% optimize.overlap_opt_shard_in_pipeline : 0.000017s : 0.00% optimize.grouped_pairwise_exchange_alltoall : 0.000018s : 0.00% optimize.overlap_recompute_and_grad_model_parallel : 0.000002s : 0.00% optimize.overlap_grad_matmul_and_grad_allreduce : 0.000012s : 0.00% optimize.split_matmul_comm_elemetwise : 0.000031s : 0.00% optimize.split_layernorm_comm : 0.000002s : 0.00% optimize.process_send_recv_for_ge : 0.000003s : 0.00% optimize.handle_group_info : 0.000001s : 0.00% auto_monad_reorder : 0.000363s : 0.05% get_jit_bprop_graph : 0.000029s : 0.00% eliminate_special_op_node : 0.000992s : 0.13% validate : 0.000299s : 0.04% distribtued_split : 0.000454s : 0.06% task_emit : 0.092602s : 12.53% execute : 0.000031s : 0.00% Time group info: ------[substitution.] 0.247733 4489 0.09% : 0.000212s : 57: substitution.arithmetic_simplify 0.02% : 0.000040s : 16: substitution.cast_eliminate 0.02% : 0.000042s : 55: substitution.depend_value_elim 0.01% : 0.000016s : 16: substitution.environ_get_add_eliminate 0.00% : 0.000007s : 8: substitution.environ_get_depend_swap 0.01% : 0.000020s : 32: substitution.environ_get_eliminate 0.02% : 0.000039s : 16: substitution.environ_get_set_eliminate 0.01% : 0.000037s : 94: substitution.float_depend_g_call 0.00% : 0.000008s : 16: substitution.float_environ_get_switch 0.01% : 0.000013s : 14: substitution.float_tuple_getitem_switch 94.67% : 0.234518s : 233: substitution.getattr_setattr_resolve 0.02% : 0.000051s : 120: substitution.graph_param_transform 0.00% : 0.000007s : 20: substitution.incorporate_call 0.00% : 0.000005s : 20: substitution.incorporate_call_switch 4.07% : 0.010094s : 377: substitution.inline 0.01% : 0.000013s : 34: substitution.less_batch_normalization 0.01% : 0.000031s : 160: substitution.load_eliminater 0.16% : 0.000396s : 505: substitution.meta_unpack_prepare 0.02% : 0.000057s : 72: substitution.minmaximum_grad 0.01% : 0.000017s : 8: substitution.partial_defer_inline 0.05% : 0.000123s : 94: substitution.partial_eliminate 0.00% : 0.000009s : 120: substitution.partial_unused_args_eliminate 0.02% : 0.000043s : 31: substitution.real_op_eliminate 0.01% : 0.000013s : 32: substitution.reduce_all_const_elim 0.01% : 0.000032s : 338: substitution.remove_not_recompute_node 0.11% : 0.000280s : 264: substitution.replace_applicator 0.01% : 0.000031s : 142: substitution.replace_old_param 0.00% : 0.000011s : 2: substitution.reshape_eliminate 0.00% : 0.000005s : 10: substitution.set_cell_output_no_recompute 0.00% : 0.000006s : 2: substitution.specialize_transform 0.01% : 0.000020s : 32: substitution.split_environ_get_set_with_tuple_value 0.01% : 0.000025s : 31: substitution.switch_simplify 0.07% : 0.000185s : 76: substitution.tuple_list_convert_item_index_to_positive 0.04% : 0.000095s : 92: substitution.tuple_list_get_item_const_eliminator 0.06% : 0.000160s : 92: substitution.tuple_list_get_item_depend_reorder 0.19% : 0.000470s : 283: substitution.tuple_list_get_item_eliminator 0.05% : 0.000116s : 92: substitution.tuple_list_get_set_item_eliminator 0.08% : 0.000201s : 416: substitution.updatestate_pure_node_eliminater 0.12% : 0.000286s : 467: substitution.updatestate_useless_node_eliminater ------[renormalize.] 0.061077 4 59.22% : 0.036168s : 2: renormalize.infer 40.78% : 0.024908s : 2: renormalize.specialize ------[replace.] 0.008077 886 0.07% : 0.000005s : 1: replace.arithmetic_simplify 1.00% : 0.000081s : 16: replace.cast_eliminate 0.63% : 0.000051s : 10: replace.depend_value_elim 0.99% : 0.000080s : 8: replace.environ_get_set_eliminate 37.00% : 0.002988s : 210: replace.getattr_setattr_resolve 30.02% : 0.002425s : 341: replace.inline 0.39% : 0.000032s : 1: replace.meta_unpack_prepare 4.92% : 0.000397s : 32: replace.partial_eliminate 1.82% : 0.000147s : 31: replace.real_op_eliminate 1.92% : 0.000155s : 9: replace.replace_applicator 3.41% : 0.000275s : 31: replace.switch_simplify 1.31% : 0.000106s : 16: replace.tuple_list_get_item_depend_reorder 16.27% : 0.001314s : 179: replace.tuple_list_get_item_eliminator 0.25% : 0.000021s : 1: replace.updatestate_useless_node_eliminater ------[match.] 0.244669 886 0.00% : 0.000011s : 1: match.arithmetic_simplify 0.02% : 0.000040s : 16: match.cast_eliminate 0.00% : 0.000007s : 10: match.depend_value_elim 0.01% : 0.000033s : 8: match.environ_get_set_eliminate 95.62% : 0.233962s : 210: match.getattr_setattr_resolve 4.04% : 0.009887s : 341: match.inline 0.07% : 0.000166s : 1: match.meta_unpack_prepare 0.04% : 0.000095s : 32: match.partial_eliminate 0.02% : 0.000043s : 31: match.real_op_eliminate 0.01% : 0.000031s : 9: match.replace_applicator 0.01% : 0.000025s : 31: match.switch_simplify 0.02% : 0.000055s : 16: match.tuple_list_get_item_depend_reorder 0.12% : 0.000306s : 179: match.tuple_list_get_item_eliminator 0.00% : 0.000007s : 1: match.updatestate_useless_node_eliminater ------[func_graph_cloner_run.] 0.040896 648 72.24% : 0.029542s : 267: func_graph_cloner_run.FuncGraphClonerGraph 27.76% : 0.011354s : 381: func_graph_cloner_run.FuncGraphSpecializer ------[meta_graph.] 0.000000 0 ------[manager.] 0.000000 0 ------[pynative] 0.000000 0 ------[others.] 0.284246 170 3.92% : 0.011145s : 103: opt.transform.opt_a 0.42% : 0.001200s : 23: opt.transform.opt_b 84.44% : 0.240030s : 6: opt.transform.opt_resolve 0.36% : 0.001017s : 1: opt.transforms.meta_unpack_prepare 10.58% : 0.030066s : 30: opt.transforms.opt_a 0.06% : 0.000164s : 1: opt.transforms.opt_after_cconv 0.02% : 0.000052s : 1: opt.transforms.opt_b 0.14% : 0.000403s : 2: opt.transforms.opt_trans_graph 0.06% : 0.000171s : 3: opt.transforms.special_op_eliminate [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.235.178 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1385] Run] End [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.235.208 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:846] SaveCompiledGraph] Save compiled func graph(381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316) phase(train.1704857851317627904.281471260670096.0)! [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.235.229 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:864] SaveCompiledGraph] End save compiled func graph! [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.235.242 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:942] CleanCompileRes] Clean compile resource start [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.245.369 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:956] CleanCompileRes] Clean compile resource end [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.245.405 [mindspore/ccsrc/pipeline/jit/ps/event_message_print.cc:37] PrintEventMessage] End compiling '_DataWrapper.construct'. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.245.419 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1039] CompileInner] Finish compiling. [WARNING] ME(164039:281472963273792,MainProcess):2024-01-10-11:37:32.246.823 [mindspore/parallel/_utils.py:259] You are suggested to use mindspore.context.set_auto_parallel_context(parameter_broadcast=True) or mindspore.common.set_seed() to share parameters among multi-devices. [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:37:32.248.482 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:37:32.248.559 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:37:32.248.615 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_1 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.248.888 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode -1 [INFO] PRE_ACT(164039,fffe1cf490f0,python):2024-01-10-11:37:32.248.951 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:237] SetMemAllocUintSize] Set mem alloc unit size, common 1073741824 persistent 1073741824 [INFO] DEVICE(164039,fffe1cf490f0,python):2024-01-10-11:37:32.248.971 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_pool.cc:124] AllocDeviceMem] Malloc Memory for Pool, size: 1073741824 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.249.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Float32, value=0), output index: 0 device address:0x4f997f20 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.249.434 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode (16, 6, 5, 5) [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.249.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Float32, value=0.25), output index: 0 device address:0x4b0d33a0 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.249.679 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Int64, value=10), output index: 0 device address:0x4eea76d0 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.249.799 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Float32, value=1), output index: 0 device address:0x4ae90d10 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.249.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[1], dtype=Int64, value=[32]), output index: 0 device address:0x4ae9f5c0 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.249.973 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode (6, 1, 5, 5) [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.250.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode (32, 16, 5, 5) [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.250.122 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode 0 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.250.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[32, 1], dtype=Float32, value=[...]), output index: 0 device address:0x4ae16ea0 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.250.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), output index: 0 device address:0x4af0ba30 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.250.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode (32, 6, 14, 14) [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.250.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode 1 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.250.515 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10]), output index: 0 device address:0x4afab020 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.250.597 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10]), output index: 0 device address:0x4b00b8f0 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.250.687 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[1], dtype=Int32, value=[1]), output index: 0 device address:0x4af25d10 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.250.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode false [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.250.831 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode true [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.250.926 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc3.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.251.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc3.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.251.141 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc2.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.251.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc2.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.251.335 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc1.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.251.424 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc1.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.251.526 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_conv2.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.251.618 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_conv1.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.251.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_global_step, device type:2 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.251.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc3.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.251.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_learning_rate, device type:2 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.251.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_momentum, device type:2 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.252.071 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc3.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.252.163 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc2.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.252.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc2.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.252.361 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc1.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.252.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc1.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.252.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.conv2.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:37:32.252.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.conv1.weight, device type:2 [INFO] PRE_ACT(164039,fffe1cf490f0,python):2024-01-10-11:37:32.252.765 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:237] SetMemAllocUintSize] Set mem alloc unit size, common 1073741824 persistent 1073741824 [INFO] DEVICE(164039,fffe1cf490f0,python):2024-01-10-11:37:32.252.786 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_pool.cc:124] AllocDeviceMem] Malloc Memory for Pool, size: 1073741824 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:37:32.252.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] GE(164039,python):2024-01-10-11:37:32.337.507 [graph_var_manager.cc:1424][EVENT]167202 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:32.337.582 [graph_manager.cc:1248][EVENT]167202 PreRun:PreRun start: graph node size 2, session id 2, graph id 1, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:32.337.947 [atrace_api.c:28](tid:167202) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:32.337.977 [trace_rb_log.c:84](tid:167202) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:32.337.990 [atrace_api.c:32](tid:167202) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:32.338.010 [client_manager.cpp:157][SetProfilingCallback][tid:167202] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:32.338.447 [parallel_partitioner.cc:165][EVENT]167202 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.338.484 [parallel_partitioner.cc:178][EVENT]167202 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.338.530 [graph_prepare.cc:1378][EVENT]167202 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.338.736 [graph_manager.cc:1050][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [222] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.338.776 [graph_manager.cc:1052][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.338.850 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.338.880 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.338.967 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [74] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.338.981 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.339.028 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.339.042 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.339.053 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.339.138 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.339.158 [graph_manager.cc:1054][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [368] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.339.396 [graph_manager.cc:1055][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [223] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.340.250 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.340.275 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.340.287 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.340.296 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferShapePass is [260] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.340.306 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.340.314 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.340.323 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [46] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.340.332 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.340.340 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.341.442 [graph_manager.cc:1056][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2028] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.341.500 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.341.528 [graph_prepare.cc:1982][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [55] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.341.929 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.341.953 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.341.963 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.341.972 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferShapePass is [141] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.341.981 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.341.989 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.341.998 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.342.006 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [7] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.342.014 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferValuePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.342.052 [graph_prepare.cc:1983][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [510] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.342.075 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.342.087 [graph_prepare.cc:1984][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.342.101 [graph_prepare.cc:1985][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.342.117 [graph_prepare.cc:1986][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.342.127 [graph_prepare.cc:1987][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.342.142 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.342.153 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.342.167 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.342.238 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.342.250 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.342.259 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.342.278 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.342.287 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.342.296 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.342.304 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.342.313 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.342.321 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.342.329 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.342.337 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.342.345 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.342.354 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.342.362 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.342.370 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.342.378 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.342.399 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.342.412 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.342.442 [graph_prepare.cc:1988][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [305] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.342.455 [graph_manager.cc:1065][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [983] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.354.248 [graph_manager.cc:1077][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11773] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.354.313 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.354.363 [graph_manager.cc:1080][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [79] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.356.715 [graph_manager.cc:1081][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [2336] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.356.755 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.356.781 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.356.793 [graph_manager.cc:1082][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [47] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.356.823 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.356.837 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.356.850 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.356.880 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.356.894 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.356.906 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.356.918 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.356.954 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [26] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.356.971 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.006 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [26] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.031 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.045 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.057 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.066 [graph_manager.cc:2700][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [247] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.149 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.357.162 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.357.171 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.357.180 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.357.188 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.357.197 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CastRemovePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.357.205 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.357.219 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.357.228 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.357.236 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.357.244 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [6] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.357.253 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.357.261 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [7] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.357.269 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.357.277 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.357.287 [graph_manager.cc:2741][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [203] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.296 [graph_manager.cc:2752][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.319 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.330 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.346 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.360 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.371 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.384 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.402 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.416 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.429 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.439 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.456 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.467 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.489 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.502 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.512 [graph_manager.cc:2810][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [197] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.535 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.357.547 [graph_manager.cc:2821][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [26] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.574 [graph_manager.cc:1087][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [762] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.738 [graph_manager.cc:1088][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [151] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.781 [graph_manager.cc:1089][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.798 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.812 [graph_manager.cc:1097][EVENT]167202 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:32.357.834 [graph_manager.cc:3325][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.949 [engine_place.cc:144][EVENT]167202 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.357.966 [engine_place.cc:144][EVENT]167202 Run:The time cost of aicpu_ascend_kernel::CheckSupported is [44] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.358.035 [graph_manager.cc:3351][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [188] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.358.051 [graph_manager.cc:3364][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.358.108 [engine_partitioner.cc:1139][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.358.124 [engine_partitioner.cc:1142][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.358.246 [engine_partitioner.cc:1148][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [112] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.358.272 [engine_partitioner.cc:1155][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.358.311 [engine_partitioner.cc:1164][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [28] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.358.345 [graph_manager.cc:3405][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [282] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.358.375 [graph_manager.cc:3412][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.167 [graph_manager.cc:3422][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [1777] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.198 [graph_manager.cc:3428][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.314 [graph_manager.cc:3467][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [96] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.331 [graph_manager.cc:3377][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [2268] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.346 [graph_manager.cc:1106][EVENT]167202 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [2519] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.358 [graph_manager.cc:1115][EVENT]167202 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:32.360.380 [graph_manager.cc:1130][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.412 [graph_manager.cc:1131][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.434 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.451 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.460 [graph_manager.cc:2837][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [34] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.542 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.360.554 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.360.563 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.360.572 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.360.581 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [23] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.360.589 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:37:32.360.599 [graph_manager.cc:2864][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [123] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.611 [graph_manager.cc:2872][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.629 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.643 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.663 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.677 [compile_nodes_pass.cc:88][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.686 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.697 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.776 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [70] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.800 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.814 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.826 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.839 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.848 [graph_manager.cc:2927][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [222] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.861 [graph_manager.cc:2937][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.888 [graph_manager.cc:2943][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.360.903 [graph_manager.cc:2950][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.361.078 [graph_manager.cc:2958][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [33] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.361.108 [graph_manager.cc:1132][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [683] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.361.202 [graph_manager.cc:1135][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [81] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.361.232 [graph_manager.cc:2975][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.361.334 [graph_manager.cc:2981][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [89] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.361.350 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.361.361 [graph_manager.cc:2986][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.361.378 [graph_manager.cc:1136][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [159] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.361.478 [graph_manager.cc:3555][EVENT]167202 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [73] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.361.533 [engine_partitioner.cc:1139][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.361.550 [engine_partitioner.cc:1142][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.361.632 [engine_partitioner.cc:1148][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [73] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.361.653 [engine_partitioner.cc:1155][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.361.684 [engine_partitioner.cc:1164][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.361.721 [graph_builder.cc:865][EVENT]167202 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [213] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.361.812 [graph_builder.cc:288][EVENT]167202 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [74] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.362.000 [graph_builder.cc:293][EVENT]167202 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [173] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.362.179 [model_builder.cc:1133][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [89] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.362.420 [block_mem_assigner.cc:4069][EVENT]167527 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164039,python):2024-01-10-11:37:32.362.422 [block_mem_assigner.cc:4069][EVENT]167528 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164039,python):2024-01-10-11:37:32.362.860 [graph_mem_assigner.cc:2166][EVENT]167202 SetInputOffset:[IMAS]AfterAssignMemory : online_1 memoffset[132096], memtype[2], theory_min[264192], zero_copy[132096], total_size[132096], no_reuse[132096], streams[1], topo_mode[DFS], mop[], io_reuse[0:0], alloc_mode[] [INFO] GE(164039,python):2024-01-10-11:37:32.362.947 [model_builder.cc:1144][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [747] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.362.970 [model_builder.cc:1152][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.362.985 [model_builder.cc:1157][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.363.095 [model_builder.cc:1167][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [97] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.363.113 [model_builder.cc:1174][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.363.134 [model_builder.cc:1180][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.363.168 [model_builder.cc:1184][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.363.194 [graph_builder.cc:304][EVENT]167202 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [1173] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:32.363.286 [logger.cc:1071] 167202 ModelBindStream: model_id=64, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:37:32.363.351 [task_generator.cc:804][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.363.408 [task_generator.cc:805][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [42] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.363.879 [task_generator.cc:814][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [458] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.363.893 [task_generator.cc:954][EVENT]167202 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [547] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.363.950 [task_generator.cc:967][EVENT]167202 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [33] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:32.363.968 [logger.cc:1084] 167202 ModelUnbindStream: model_id=64, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:37:32.364.023 [graph_builder.cc:310][EVENT]167202 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [816] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.364.125 [graph_manager.cc:1152][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2727] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.364.142 [graph_manager.cc:1164][EVENT]167202 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:32.364.174 [graph_manager.cc:1271][EVENT]167202 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [25813] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.364.185 [graph_manager.cc:1272][EVENT]167202 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:37:32.364.492 [atrace_api.c:93](tid:167202) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:37:32.364.507 [atrace_api.c:95](tid:167202) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:37:32.365.146 [model_introduction.cc:236][EVENT]167202 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(164039,python):2024-01-10-11:37:32.365.166 [model_introduction.cc:294][EVENT]167202 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(164039,python):2024-01-10-11:37:32.365.180 [model_introduction.cc:366][EVENT]167202 Data:model io_info size:114 [INFO] GE(164039,python):2024-01-10-11:37:32.368.676 [graph_converter.cc:838][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1334] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.368.902 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [183] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.369.319 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [395] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.369.401 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [60] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.369.417 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [77] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.369.458 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [30] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.369.496 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.369.525 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.369.590 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [55] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.369.651 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [49] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.369.660 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [59] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.369.705 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.369.732 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.369.746 [graph_converter.cc:849][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1033] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.369.945 [graph_converter.cc:853][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [190] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.370.571 [graph_converter.cc:857][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [609] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.370.681 [graph_converter.cc:862][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [87] micro second. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:37:32.375.729 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 101 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] GE(164039,python):2024-01-10-11:37:32.451.263 [graph_var_manager.cc:1424][EVENT]167201 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:32.451.348 [graph_manager.cc:1248][EVENT]167201 PreRun:PreRun start: graph node size 6, session id 3, graph id 2, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:32.452.269 [atrace_api.c:28](tid:167201) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:32.452.347 [trace_rb_log.c:84](tid:167201) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:32.452.360 [atrace_api.c:32](tid:167201) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:32.452.379 [client_manager.cpp:157][SetProfilingCallback][tid:167201] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:32.453.319 [parallel_partitioner.cc:165][EVENT]167201 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.453.359 [parallel_partitioner.cc:178][EVENT]167201 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.453.408 [graph_prepare.cc:1378][EVENT]167201 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.454.107 [graph_manager.cc:1050][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [718] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.454.138 [graph_manager.cc:1052][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.454.337 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.454.367 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.454.415 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.454.428 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.454.474 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.454.488 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.454.505 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.454.600 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.454.620 [graph_manager.cc:1054][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [455] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.454.850 [graph_manager.cc:1055][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [217] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.456.284 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:37:32.456.312 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.456.323 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.456.333 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [467] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.456.342 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.456.351 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:37:32.456.360 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [69] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.456.368 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [21] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.456.377 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [7] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.458.499 [graph_manager.cc:1056][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3630] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.458.567 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.458.586 [graph_prepare.cc:1982][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [56] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.459.166 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:37:32.459.189 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.200 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.209 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [335] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.218 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.227 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:37:32.459.236 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [9] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.244 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.252 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.279 [graph_prepare.cc:1983][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [670] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.459.303 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.459.315 [graph_prepare.cc:1984][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.459.329 [graph_prepare.cc:1985][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.459.348 [graph_prepare.cc:1986][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.459.360 [graph_prepare.cc:1987][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.459.376 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.459.387 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.459.401 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.459.506 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [4] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.517 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.526 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrintOpPass is [6] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.535 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.543 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DropOutPass is [0] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.559 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.568 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.576 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.585 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.593 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.601 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [3] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.610 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.618 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.626 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [7] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.634 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.643 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.459.666 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.459.680 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.459.717 [graph_prepare.cc:1988][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [346] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.459.730 [graph_manager.cc:1065][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1200] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.472.477 [graph_manager.cc:1077][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12727] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.472.546 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.472.593 [graph_manager.cc:1080][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [81] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.479.755 [graph_manager.cc:1081][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [7146] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.479.808 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.479.824 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.479.836 [graph_manager.cc:1082][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.479.882 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.479.896 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.479.910 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.479.941 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.479.956 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.479.971 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.479.984 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.029 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [34] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.047 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.078 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.109 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.125 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.137 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.147 [graph_manager.cc:2700][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [271] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.294 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.480.307 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.480.317 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.480.326 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.480.334 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.480.343 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.480.351 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.480.359 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.480.374 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [1] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.480.383 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.480.391 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.480.400 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.480.408 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.480.416 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.480.424 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.480.434 [graph_manager.cc:2741][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [269] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.443 [graph_manager.cc:2752][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.466 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.478 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.496 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.510 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.522 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.534 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.552 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.565 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.578 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.588 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.601 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.612 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.631 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.644 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.663 [graph_manager.cc:2810][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [201] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.696 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.480.708 [graph_manager.cc:2821][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.737 [graph_manager.cc:1087][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [868] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.870 [graph_manager.cc:1088][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [120] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.913 [graph_manager.cc:1089][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.930 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.480.946 [graph_manager.cc:1097][EVENT]167201 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:32.480.968 [graph_manager.cc:3325][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.482.120 [engine_place.cc:144][EVENT]167201 Run:The time cost of AIcoreEngine::CheckSupported is [1018] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.482.147 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.482.157 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.482.307 [graph_manager.cc:3351][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [1325] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.482.326 [graph_manager.cc:3364][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.482.393 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.482.410 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.482.623 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [203] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.482.673 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.482.724 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.482.759 [graph_manager.cc:3405][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [420] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.482.778 [graph_manager.cc:3412][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.492.916 [graph_manager.cc:3422][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [10115] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.492.956 [graph_manager.cc:3428][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.113 [graph_manager.cc:3467][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [135] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.132 [graph_manager.cc:3377][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [10793] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.148 [graph_manager.cc:1106][EVENT]167201 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [12185] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.161 [graph_manager.cc:1115][EVENT]167201 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:32.493.185 [graph_manager.cc:1130][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.217 [graph_manager.cc:1131][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.242 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.259 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.269 [graph_manager.cc:2837][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.362 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.493.374 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [0] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.493.384 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.493.393 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.493.402 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [7] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.493.410 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.493.420 [graph_manager.cc:2864][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [134] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.432 [graph_manager.cc:2872][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.450 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.463 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.486 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.500 [compile_nodes_pass.cc:88][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.510 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.520 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.614 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [85] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.643 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.657 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.671 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.718 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.739 [graph_manager.cc:2927][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [293] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.753 [graph_manager.cc:2937][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.768 [graph_manager.cc:2943][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.779 [graph_manager.cc:2950][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.493.973 [graph_manager.cc:2958][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [44] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.494.005 [graph_manager.cc:1132][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [775] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.494.074 [graph_manager.cc:1135][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [56] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.494.114 [graph_manager.cc:2975][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.494.146 [graph_manager.cc:2981][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.494.161 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.494.170 [graph_manager.cc:2986][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.494.179 [graph_manager.cc:1136][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [88] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.494.320 [graph_manager.cc:3555][EVENT]167201 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [98] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.494.423 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.494.440 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.494.624 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [173] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.494.663 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [27] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.494.705 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [30] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.494.732 [graph_builder.cc:865][EVENT]167201 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [351] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:32.495.199 [logger.cc:1071] 167201 ModelBindStream: model_id=832, stream_id=1089, flag=0. [INFO] GE(164039,python):2024-01-10-11:37:32.495.233 [task_generator.cc:804][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [174] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.495.306 [task_generator.cc:805][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [60] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.496.086 [task_generator.cc:814][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [766] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.496.100 [task_generator.cc:954][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1041] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.496.165 [task_generator.cc:967][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [38] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:32.496.184 [logger.cc:1084] 167201 ModelUnbindStream: model_id=832, stream_id=1089, [INFO] GE(164039,python):2024-01-10-11:37:32.496.391 [graph_manager.cc:1152][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2177] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.496.410 [graph_manager.cc:1164][EVENT]167201 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:32.496.443 [graph_manager.cc:1271][EVENT]167201 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [43224] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.496.454 [graph_manager.cc:1272][EVENT]167201 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:37:32.496.761 [atrace_api.c:93](tid:167201) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:37:32.496.778 [atrace_api.c:95](tid:167201) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:37:32.503.474 [graph_converter.cc:838][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [2087] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.503.642 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [124] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.504.284 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [621] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.504.573 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [257] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.504.593 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [279] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.504.854 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [249] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.504.899 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [27] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.504.934 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.505.173 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [227] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.505.273 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [81] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.505.287 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [95] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.505.321 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [25] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.505.350 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.505.380 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.505.479 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [90] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.505.561 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [70] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.505.572 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [82] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.505.603 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.505.630 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.505.643 [graph_converter.cc:849][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [2131] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.505.948 [graph_converter.cc:853][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [296] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.506.878 [graph_converter.cc:857][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [912] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.507.066 [graph_converter.cc:862][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [164] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.570.767 [graph_var_manager.cc:1424][EVENT]167202 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:32.570.858 [graph_manager.cc:1248][EVENT]167202 PreRun:PreRun start: graph node size 4, session id 4, graph id 3, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:32.571.163 [atrace_api.c:28](tid:167202) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:32.571.208 [trace_rb_log.c:84](tid:167202) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:32.571.222 [atrace_api.c:32](tid:167202) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:32.571.240 [client_manager.cpp:157][SetProfilingCallback][tid:167202] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:32.571.628 [parallel_partitioner.cc:165][EVENT]167202 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.571.663 [parallel_partitioner.cc:178][EVENT]167202 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.571.710 [graph_prepare.cc:1378][EVENT]167202 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.571.893 [graph_manager.cc:1050][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [200] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.571.918 [graph_manager.cc:1052][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.572.057 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.572.087 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.572.137 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.572.150 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.572.196 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.572.210 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.572.229 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.572.317 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.572.338 [graph_manager.cc:1054][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [408] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.572.572 [graph_manager.cc:1055][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [219] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.573.680 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:32.573.741 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.573.753 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.573.763 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferShapePass is [403] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.573.772 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.573.789 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:32.573.799 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.573.807 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.573.816 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.576.964 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:32.576.993 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.577.004 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.577.014 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferShapePass is [369] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.577.023 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.577.032 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:32.577.040 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.577.048 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.577.057 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.578.287 [graph_manager.cc:1056][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [5695] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.578.356 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.578.374 [graph_prepare.cc:1982][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [52] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.578.915 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:32.578.941 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.578.952 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.578.962 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferShapePass is [299] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.578.971 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.578.980 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:32.578.988 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.579.006 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.579.015 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.579.063 [graph_prepare.cc:1983][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [675] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.579.088 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.579.101 [graph_prepare.cc:1984][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.579.115 [graph_prepare.cc:1985][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.579.131 [graph_prepare.cc:1986][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.579.141 [graph_prepare.cc:1987][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.579.155 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.579.166 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.579.179 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.579.269 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.579.281 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.579.292 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.579.303 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.579.312 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.579.320 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.579.329 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.579.337 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.579.345 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.579.353 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.579.362 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.579.370 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SnapshotPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.579.384 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.579.393 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.579.401 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.579.410 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:32.579.433 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.579.445 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.579.479 [graph_prepare.cc:1988][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [328] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.579.491 [graph_manager.cc:1065][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1170] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.591.441 [graph_manager.cc:1077][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11930] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.591.545 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.591.592 [graph_manager.cc:1080][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [113] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.601.343 [graph_manager.cc:1081][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [9737] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.601.401 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.601.418 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.601.434 [graph_manager.cc:1082][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [42] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.601.467 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.601.484 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.601.499 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.601.672 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [163] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.601.714 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.601.823 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [97] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.601.864 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.601.916 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [40] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.601.939 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.601.959 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.090 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [120] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.112 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.128 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.138 [graph_manager.cc:2700][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [678] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.468 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of EnterPass is [4] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.602.487 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.602.497 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [7] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.602.510 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.602.518 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [6] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.602.527 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CastRemovePass is [47] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.602.536 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [6] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.602.544 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.602.553 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [13] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.602.561 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.602.569 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [17] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.602.580 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [80] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.602.591 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.602.600 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [12] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.602.611 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.602.632 [graph_manager.cc:2741][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [474] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.642 [graph_manager.cc:2752][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.668 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.680 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.703 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.720 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.735 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.747 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.768 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.782 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.795 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.805 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.821 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.835 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.860 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.873 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.882 [graph_manager.cc:2810][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [220] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.928 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.602.943 [graph_manager.cc:2821][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [52] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.602.974 [graph_manager.cc:1087][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1521] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.603.566 [graph_manager.cc:1088][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [577] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.603.628 [graph_manager.cc:1089][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [32] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.603.658 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.603.676 [graph_manager.cc:1097][EVENT]167202 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:32.603.699 [graph_manager.cc:3325][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.613.068 [engine_place.cc:144][EVENT]167202 Run:The time cost of AIcoreEngine::CheckSupported is [9114] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.613.100 [engine_place.cc:144][EVENT]167202 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.613.111 [engine_place.cc:144][EVENT]167202 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.613.208 [graph_manager.cc:3351][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [9492] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.613.228 [graph_manager.cc:3364][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.613.306 [engine_partitioner.cc:1139][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [29] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.613.338 [engine_partitioner.cc:1142][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.613.518 [engine_partitioner.cc:1148][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [166] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.613.566 [engine_partitioner.cc:1155][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [30] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.613.618 [engine_partitioner.cc:1164][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.613.655 [graph_manager.cc:3405][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [414] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.613.674 [graph_manager.cc:3412][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.798.514 [graph_manager.cc:3422][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [184811] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.798.567 [graph_manager.cc:3428][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.798.755 [graph_manager.cc:3467][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [165] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.798.777 [graph_manager.cc:3377][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [185536] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.798.794 [graph_manager.cc:1106][EVENT]167202 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [195102] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.798.819 [graph_manager.cc:1115][EVENT]167202 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:32.798.847 [graph_manager.cc:1130][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.798.887 [graph_manager.cc:1131][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.798.918 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.798.942 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.798.953 [graph_manager.cc:2837][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [49] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.102 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [30] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.799.118 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.799.128 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondRemovePass is [7] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.799.136 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.799.145 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [11] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.799.154 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [16] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:32.799.165 [graph_manager.cc:2864][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [193] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.178 [graph_manager.cc:2872][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.198 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.212 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.229 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.243 [compile_nodes_pass.cc:88][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.253 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.263 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.373 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [100] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.429 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [42] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.450 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.465 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.479 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.489 [graph_manager.cc:2927][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [294] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.502 [graph_manager.cc:2937][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.517 [graph_manager.cc:2943][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.531 [graph_manager.cc:2950][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.755 [graph_manager.cc:2958][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [58] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.791 [graph_manager.cc:1132][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [890] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.877 [graph_manager.cc:1135][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [72] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.914 [graph_manager.cc:2975][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.946 [graph_manager.cc:2981][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.961 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.970 [graph_manager.cc:2986][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.799.979 [graph_manager.cc:1136][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [87] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.800.349 [graph_manager.cc:3555][EVENT]167202 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [329] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.800.488 [engine_partitioner.cc:1139][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [34] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.800.522 [engine_partitioner.cc:1142][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.800.668 [engine_partitioner.cc:1148][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [136] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.800.704 [engine_partitioner.cc:1155][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.800.757 [engine_partitioner.cc:1164][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [34] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.800.785 [graph_builder.cc:865][EVENT]167202 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [363] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:32.801.201 [logger.cc:1071] 167202 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:37:32.801.232 [task_generator.cc:804][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [91] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.801.320 [task_generator.cc:805][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [76] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.803.161 [task_generator.cc:814][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1826] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.803.181 [task_generator.cc:954][EVENT]167202 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2041] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.803.257 [task_generator.cc:967][EVENT]167202 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [41] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:32.803.284 [logger.cc:1084] 167202 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:37:32.804.245 [graph_manager.cc:1152][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4231] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.804.281 [graph_manager.cc:1164][EVENT]167202 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:32.804.320 [graph_manager.cc:1271][EVENT]167202 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [232781] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.804.331 [graph_manager.cc:1272][EVENT]167202 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:37:32.804.653 [atrace_api.c:93](tid:167202) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:37:32.804.672 [atrace_api.c:95](tid:167202) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:37:32.829.802 [graph_converter.cc:838][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [9543] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.830.038 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [176] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.831.688 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [1626] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.832.105 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [388] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.832.130 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [415] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.832.407 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [264] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.832.495 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [64] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.832.568 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [56] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.833.046 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [463] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.833.279 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [207] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.833.315 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [244] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.833.388 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [61] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.833.451 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [51] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.833.515 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [52] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.833.772 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [246] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.833.987 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [196] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.834.006 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [216] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.834.075 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [58] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.834.137 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [50] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.834.156 [graph_converter.cc:849][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4299] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.834.921 [graph_converter.cc:853][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [754] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.837.026 [graph_converter.cc:857][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2083] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.837.439 [graph_converter.cc:862][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [385] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.926.014 [graph_var_manager.cc:1424][EVENT]167201 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:32.926.105 [graph_manager.cc:1248][EVENT]167201 PreRun:PreRun start: graph node size 3, session id 5, graph id 4, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:32.926.414 [atrace_api.c:28](tid:167201) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:32.926.460 [trace_rb_log.c:84](tid:167201) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:32.926.477 [atrace_api.c:32](tid:167201) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:32.926.495 [client_manager.cpp:157][SetProfilingCallback][tid:167201] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:32.926.927 [parallel_partitioner.cc:165][EVENT]167201 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.926.963 [parallel_partitioner.cc:178][EVENT]167201 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.927.008 [graph_prepare.cc:1378][EVENT]167201 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.927.186 [graph_manager.cc:1050][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [194] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.927.225 [graph_manager.cc:1052][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.927.350 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.927.379 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.927.428 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.927.442 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.927.488 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.927.502 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.927.519 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.927.604 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.927.625 [graph_manager.cc:1054][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [387] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.927.869 [graph_manager.cc:1055][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [229] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.928.763 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.928.788 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.928.800 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.928.809 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [260] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.928.818 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.928.827 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.928.836 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [62] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.928.844 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.928.853 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.930.572 [graph_manager.cc:1056][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2684] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.930.636 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.930.663 [graph_prepare.cc:1982][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [59] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.931.035 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.931.056 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.066 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.076 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [179] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.084 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.093 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:32.931.102 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.110 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.118 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.160 [graph_prepare.cc:1983][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [483] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.931.184 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.931.195 [graph_prepare.cc:1984][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.931.208 [graph_prepare.cc:1985][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.931.222 [graph_prepare.cc:1986][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.931.233 [graph_prepare.cc:1987][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.931.246 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.931.258 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.931.272 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.931.352 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.364 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.373 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.381 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.396 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.406 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.414 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.422 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.431 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.439 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.448 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.456 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.464 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.472 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.480 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.488 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.931.510 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.931.523 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.931.553 [graph_prepare.cc:1988][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [311] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.931.566 [graph_manager.cc:1065][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [962] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.943.362 [graph_manager.cc:1077][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11776] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.943.428 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.943.473 [graph_manager.cc:1080][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [77] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.946.574 [graph_manager.cc:1081][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3085] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.946.614 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.946.629 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.946.657 [graph_manager.cc:1082][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [51] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.946.690 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.946.704 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.946.718 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.946.750 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.946.764 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.946.778 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.946.790 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.946.828 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [28] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.946.846 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.946.863 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.946.888 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.946.902 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.946.913 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.946.921 [graph_manager.cc:2700][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [237] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.027 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.947.041 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AddNPass is [0] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.947.050 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.947.059 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.947.068 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.947.077 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CastRemovePass is [9] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.947.085 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.947.100 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.947.108 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.947.117 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.947.125 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.947.133 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.947.141 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.947.150 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.947.158 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.947.168 [graph_manager.cc:2741][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [229] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.177 [graph_manager.cc:2752][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.199 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.210 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.227 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.240 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.252 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.263 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.281 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.294 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.307 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.317 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.329 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.340 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.358 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.375 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.384 [graph_manager.cc:2810][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [189] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.410 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.947.421 [graph_manager.cc:2821][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [29] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.449 [graph_manager.cc:1087][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [772] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.578 [graph_manager.cc:1088][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [117] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.616 [graph_manager.cc:1089][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.634 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.947.649 [graph_manager.cc:1097][EVENT]167201 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:32.947.670 [graph_manager.cc:3325][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.948.012 [engine_place.cc:144][EVENT]167201 Run:The time cost of AIcoreEngine::CheckSupported is [245] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.948.035 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.948.045 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.948.118 [graph_manager.cc:3351][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [434] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.948.136 [graph_manager.cc:3364][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.948.200 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.948.217 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.948.349 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [122] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.948.390 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [28] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.948.434 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [34] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.948.468 [graph_manager.cc:3405][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [319] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.948.494 [graph_manager.cc:3412][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.956.725 [graph_manager.cc:3422][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [8216] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.956.762 [graph_manager.cc:3428][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.956.886 [graph_manager.cc:3467][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [104] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.956.903 [graph_manager.cc:3377][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [8755] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.956.918 [graph_manager.cc:1106][EVENT]167201 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [9254] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.956.930 [graph_manager.cc:1115][EVENT]167201 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:32.956.952 [graph_manager.cc:1130][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.956.984 [graph_manager.cc:1131][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.007 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.024 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.035 [graph_manager.cc:2837][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.108 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.957.119 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.957.129 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.957.138 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.957.146 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [4] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.957.155 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:32.957.164 [graph_manager.cc:2864][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [113] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.176 [graph_manager.cc:2872][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.194 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.215 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.231 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.244 [compile_nodes_pass.cc:88][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.253 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.263 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.334 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [62] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.360 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.372 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.385 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.397 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.406 [graph_manager.cc:2927][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [215] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.417 [graph_manager.cc:2937][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.431 [graph_manager.cc:2943][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.442 [graph_manager.cc:2950][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.618 [graph_manager.cc:2958][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.647 [graph_manager.cc:1132][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [650] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.726 [graph_manager.cc:1135][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [66] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.759 [graph_manager.cc:2975][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.790 [graph_manager.cc:2981][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.804 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.814 [graph_manager.cc:2986][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.830 [graph_manager.cc:1136][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [84] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.957.946 [graph_manager.cc:3555][EVENT]167201 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [85] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.958.032 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.958.048 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.958.141 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [83] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.958.171 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.958.208 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [26] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.958.229 [graph_builder.cc:865][EVENT]167201 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [229] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:32.958.508 [logger.cc:1071] 167201 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:37:32.958.537 [task_generator.cc:804][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [73] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.958.594 [task_generator.cc:805][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [46] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.959.226 [task_generator.cc:814][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [619] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.959.241 [task_generator.cc:954][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [777] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.959.303 [task_generator.cc:967][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [35] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:32.959.321 [logger.cc:1084] 167201 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:37:32.959.491 [graph_manager.cc:1152][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [1637] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.959.509 [graph_manager.cc:1164][EVENT]167201 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:32.959.541 [graph_manager.cc:1271][EVENT]167201 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [32701] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.959.551 [graph_manager.cc:1272][EVENT]167201 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:37:32.959.863 [atrace_api.c:93](tid:167201) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:37:32.959.880 [atrace_api.c:95](tid:167201) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:37:32.964.534 [graph_converter.cc:838][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1299] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.964.695 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [119] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.965.159 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [434] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.965.346 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [166] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.965.366 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [186] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.965.580 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [202] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.965.617 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.965.647 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.965.837 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [179] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.965.918 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [62] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.965.931 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [76] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.965.959 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.965.983 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.966.007 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.966.077 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [60] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.966.141 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [53] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.966.152 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [64] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.966.177 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.966.199 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.966.211 [graph_converter.cc:849][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1642] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.966.413 [graph_converter.cc:853][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [192] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.967.076 [graph_converter.cc:857][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [649] micro second. [INFO] GE(164039,python):2024-01-10-11:37:32.967.208 [graph_converter.cc:862][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [110] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.047.891 [graph_var_manager.cc:1424][EVENT]167204 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:33.047.979 [graph_manager.cc:1248][EVENT]167204 PreRun:PreRun start: graph node size 3, session id 6, graph id 5, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:33.048.890 [atrace_api.c:28](tid:167204) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:33.048.963 [trace_rb_log.c:84](tid:167204) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:33.048.977 [atrace_api.c:32](tid:167204) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:33.048.996 [client_manager.cpp:157][SetProfilingCallback][tid:167204] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:33.049.925 [parallel_partitioner.cc:165][EVENT]167204 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.049.962 [parallel_partitioner.cc:178][EVENT]167204 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.050.009 [graph_prepare.cc:1378][EVENT]167204 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.050.664 [graph_manager.cc:1050][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [672] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.050.692 [graph_manager.cc:1052][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.050.818 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.050.848 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.050.896 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.050.910 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.050.953 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.050.966 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.050.983 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.051.072 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.051.093 [graph_manager.cc:1054][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [388] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.051.334 [graph_manager.cc:1055][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [228] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.052.254 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:33.052.280 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.052.291 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.052.301 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferShapePass is [312] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.052.320 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.052.329 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:33.052.338 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [10] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.052.346 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.052.355 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.054.108 [graph_manager.cc:1056][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2755] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.054.169 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.054.187 [graph_prepare.cc:1982][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [47] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.054.592 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:33.054.614 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.054.625 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.054.635 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferShapePass is [198] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.054.644 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.054.652 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:33.054.661 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.054.669 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.054.678 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.054.718 [graph_prepare.cc:1983][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [517] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.054.742 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.054.753 [graph_prepare.cc:1984][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.054.767 [graph_prepare.cc:1985][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.054.787 [graph_prepare.cc:1986][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.054.807 [graph_prepare.cc:1987][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.054.822 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.054.833 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.054.846 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.054.928 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of EnterPass is [4] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.054.939 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.054.948 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.054.957 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.054.965 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.054.974 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.054.982 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [0] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.054.991 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.054.999 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.055.008 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.055.016 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.055.025 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.055.033 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.055.041 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.055.050 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.055.058 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.055.080 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.055.093 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.055.123 [graph_prepare.cc:1988][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [307] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.055.136 [graph_manager.cc:1065][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [997] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.066.862 [graph_manager.cc:1077][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11701] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.066.929 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.066.973 [graph_manager.cc:1080][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [76] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.071.967 [graph_manager.cc:1081][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [4978] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.013 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.029 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.041 [graph_manager.cc:1082][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.072 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.086 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.101 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.236 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [125] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.254 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.330 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [64] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.345 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.388 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [32] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.408 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.427 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.504 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [66] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.520 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.534 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.544 [graph_manager.cc:2700][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [477] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.762 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.072.776 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AddNPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.072.786 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.072.795 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [4] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.072.804 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.072.813 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CastRemovePass is [34] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.072.821 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.072.830 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.072.838 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [10] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.072.846 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.072.854 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.072.863 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.072.871 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.072.879 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [7] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.072.887 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.072.897 [graph_manager.cc:2741][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [322] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.906 [graph_manager.cc:2752][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.930 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.941 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.960 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.974 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.985 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.072.997 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.073.023 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.073.037 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.073.051 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.073.062 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.073.074 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.073.085 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.073.105 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.073.117 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.073.127 [graph_manager.cc:2810][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [201] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.073.164 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.073.175 [graph_manager.cc:2821][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.073.202 [graph_manager.cc:1087][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1142] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.073.705 [graph_manager.cc:1088][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [489] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.073.759 [graph_manager.cc:1089][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [27] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.073.780 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.073.796 [graph_manager.cc:1097][EVENT]167204 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:33.073.818 [graph_manager.cc:3325][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.080.859 [engine_place.cc:144][EVENT]167204 Run:The time cost of AIcoreEngine::CheckSupported is [6860] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.080.891 [engine_place.cc:144][EVENT]167204 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.080.901 [engine_place.cc:144][EVENT]167204 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.080.986 [graph_manager.cc:3351][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [7154] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.081.005 [graph_manager.cc:3364][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.081.089 [engine_partitioner.cc:1139][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [24] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.081.116 [engine_partitioner.cc:1142][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.081.266 [engine_partitioner.cc:1148][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [138] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.081.311 [engine_partitioner.cc:1155][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.081.357 [engine_partitioner.cc:1164][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.081.393 [graph_manager.cc:3405][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [365] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.081.413 [graph_manager.cc:3412][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.149.934 [graph_manager.cc:3422][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [68507] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.149.982 [graph_manager.cc:3428][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.132 [graph_manager.cc:3467][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [128] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.151 [graph_manager.cc:3377][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [69124] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.168 [graph_manager.cc:1106][EVENT]167204 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [76356] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.181 [graph_manager.cc:1115][EVENT]167204 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:33.150.206 [graph_manager.cc:1130][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.241 [graph_manager.cc:1131][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.270 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.290 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.300 [graph_manager.cc:2837][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [42] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.419 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.150.432 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.150.453 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.150.463 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of BitcastPass is [0] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.150.472 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.150.481 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [12] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:37:33.150.491 [graph_manager.cc:2864][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [171] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.503 [graph_manager.cc:2872][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.522 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.536 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.553 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.567 [compile_nodes_pass.cc:88][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.577 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.587 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.677 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [82] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.712 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.726 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.739 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.752 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.761 [graph_manager.cc:2927][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [243] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.773 [graph_manager.cc:2937][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.788 [graph_manager.cc:2943][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.150.799 [graph_manager.cc:2950][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.151.011 [graph_manager.cc:2958][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [48] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.151.043 [graph_manager.cc:1132][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [787] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.151.120 [graph_manager.cc:1135][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [64] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.151.161 [graph_manager.cc:2975][EVENT]167204 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [24] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.151.194 [graph_manager.cc:2981][EVENT]167204 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.151.208 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.151.218 [graph_manager.cc:2986][EVENT]167204 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.151.228 [graph_manager.cc:1136][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [92] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.151.519 [graph_manager.cc:3555][EVENT]167204 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [254] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.151.638 [engine_partitioner.cc:1139][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [26] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.151.663 [engine_partitioner.cc:1142][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.151.776 [engine_partitioner.cc:1148][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [103] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.151.808 [engine_partitioner.cc:1155][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.151.851 [engine_partitioner.cc:1164][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [32] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.151.876 [graph_builder.cc:865][EVENT]167204 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [293] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:33.152.354 [logger.cc:1071] 167204 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:37:33.152.386 [task_generator.cc:804][EVENT]167204 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [166] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.152.465 [task_generator.cc:805][EVENT]167204 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [65] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.153.765 [task_generator.cc:814][EVENT]167204 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1284] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.153.781 [task_generator.cc:954][EVENT]167204 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1563] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.153.849 [task_generator.cc:967][EVENT]167204 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [37] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:33.153.881 [logger.cc:1084] 167204 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:37:33.155.502 [graph_manager.cc:1152][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4243] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.155.535 [graph_manager.cc:1164][EVENT]167204 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:33.155.570 [graph_manager.cc:1271][EVENT]167204 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [105730] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.155.581 [graph_manager.cc:1272][EVENT]167204 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:37:33.155.905 [atrace_api.c:93](tid:167204) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:37:33.155.921 [atrace_api.c:95](tid:167204) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:37:33.173.036 [graph_converter.cc:838][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [5998] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.173.239 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of ZeroCopy is [146] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.174.331 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CEM is [1068] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.174.609 [copy_flow_launch_fuse.cc:395][EVENT]167204 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [249] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.174.633 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [275] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.174.870 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [224] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.174.932 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [42] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.174.984 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of ZeroCopy is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.175.319 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CEM is [322] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.175.480 [copy_flow_launch_fuse.cc:395][EVENT]167204 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [139] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.175.497 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [158] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.175.548 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [40] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.175.592 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [33] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.175.637 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of ZeroCopy is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.175.801 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CEM is [153] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.175.941 [copy_flow_launch_fuse.cc:395][EVENT]167204 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [127] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.175.955 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [142] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.176.002 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.176.060 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [34] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.176.078 [graph_converter.cc:849][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [2991] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.176.595 [graph_converter.cc:853][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [507] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.178.003 [graph_converter.cc:857][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1387] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.178.289 [graph_converter.cc:862][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [258] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.267.495 [graph_var_manager.cc:1424][EVENT]167203 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:33.267.584 [graph_manager.cc:1248][EVENT]167203 PreRun:PreRun start: graph node size 4, session id 7, graph id 6, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:33.268.453 [atrace_api.c:28](tid:167203) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:33.268.525 [trace_rb_log.c:84](tid:167203) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:33.268.538 [atrace_api.c:32](tid:167203) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:33.268.557 [client_manager.cpp:157][SetProfilingCallback][tid:167203] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:33.269.446 [parallel_partitioner.cc:165][EVENT]167203 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.269.488 [parallel_partitioner.cc:178][EVENT]167203 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.269.539 [graph_prepare.cc:1378][EVENT]167203 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.270.193 [graph_manager.cc:1050][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [675] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.270.225 [graph_manager.cc:1052][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.270.367 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.270.398 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.270.446 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.270.459 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.270.505 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.270.518 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.270.536 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.270.640 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.270.661 [graph_manager.cc:1054][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [424] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.270.898 [graph_manager.cc:1055][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [224] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.272.071 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:33.272.097 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.272.108 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.272.118 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferShapePass is [475] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.272.127 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.272.136 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:33.272.144 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.272.153 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.272.161 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.274.732 [graph_manager.cc:1056][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3813] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.274.799 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.274.817 [graph_prepare.cc:1982][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.275.370 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:33.275.394 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.405 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.415 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferShapePass is [344] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.424 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.433 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:33.275.441 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.461 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.470 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.496 [graph_prepare.cc:1983][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [664] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.275.520 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.275.532 [graph_prepare.cc:1984][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.275.545 [graph_prepare.cc:1985][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.275.565 [graph_prepare.cc:1986][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.275.577 [graph_prepare.cc:1987][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.275.591 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.275.601 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.275.615 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.275.705 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.716 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.725 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.733 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.742 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.750 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.758 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.767 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.775 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.783 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.791 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.799 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.813 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.822 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.830 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.838 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.275.859 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.275.872 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.275.903 [graph_prepare.cc:1988][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [318] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.275.915 [graph_manager.cc:1065][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1151] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.288.779 [graph_manager.cc:1077][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12845] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.288.874 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.288.923 [graph_manager.cc:1080][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [109] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.299.488 [graph_manager.cc:1081][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [10549] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.299.542 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.299.558 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.299.570 [graph_manager.cc:1082][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.299.601 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.299.615 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.299.630 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.299.786 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [146] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.299.803 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.299.893 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [79] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.299.923 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.299.972 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.299.993 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.013 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.101 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [77] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.119 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.131 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.141 [graph_manager.cc:2700][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [545] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.377 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.300.392 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.300.402 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.300.411 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.300.420 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.300.428 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CastRemovePass is [40] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.300.437 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.300.445 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.300.453 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [10] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.300.461 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.300.470 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.300.478 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [15] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.300.486 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.300.494 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [10] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.300.502 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [4] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.300.524 [graph_manager.cc:2741][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [364] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.534 [graph_manager.cc:2752][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.557 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.569 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.589 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.604 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.616 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.629 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.650 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.664 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.676 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.686 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.698 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.710 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.731 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.744 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.754 [graph_manager.cc:2810][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [201] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.796 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.300.807 [graph_manager.cc:2821][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [45] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.300.836 [graph_manager.cc:1087][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1247] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.301.403 [graph_manager.cc:1088][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [554] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.301.461 [graph_manager.cc:1089][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [30] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.301.490 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.301.509 [graph_manager.cc:1097][EVENT]167203 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:33.301.530 [graph_manager.cc:3325][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.310.804 [engine_place.cc:144][EVENT]167203 Run:The time cost of AIcoreEngine::CheckSupported is [9059] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.310.834 [engine_place.cc:144][EVENT]167203 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.310.845 [engine_place.cc:144][EVENT]167203 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.310.937 [graph_manager.cc:3351][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [9393] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.310.955 [graph_manager.cc:3364][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.311.032 [engine_partitioner.cc:1139][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [28] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.311.063 [engine_partitioner.cc:1142][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.311.245 [engine_partitioner.cc:1148][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [172] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.311.288 [engine_partitioner.cc:1155][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [28] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.311.339 [engine_partitioner.cc:1164][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.311.376 [graph_manager.cc:3405][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [408] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.311.394 [graph_manager.cc:3412][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.433.315 [graph_manager.cc:3422][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [121905] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.433.368 [graph_manager.cc:3428][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.433.543 [graph_manager.cc:3467][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [154] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.433.563 [graph_manager.cc:3377][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [122597] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.433.581 [graph_manager.cc:1106][EVENT]167203 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [132057] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.433.593 [graph_manager.cc:1115][EVENT]167203 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:33.433.633 [graph_manager.cc:1130][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.433.666 [graph_manager.cc:1131][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.433.732 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [46] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.433.754 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.433.764 [graph_manager.cc:2837][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [81] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.433.906 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [27] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.433.920 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.433.929 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CondRemovePass is [6] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.433.938 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.433.947 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [10] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.433.955 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [16] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.433.966 [graph_manager.cc:2864][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [183] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.433.979 [graph_manager.cc:2872][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.433.999 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.014 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.030 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.045 [compile_nodes_pass.cc:88][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.054 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.064 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.173 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [101] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.224 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.245 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.261 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.275 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.285 [graph_manager.cc:2927][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [289] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.297 [graph_manager.cc:2937][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.312 [graph_manager.cc:2943][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.323 [graph_manager.cc:2950][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.542 [graph_manager.cc:2958][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [56] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.574 [graph_manager.cc:1132][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [893] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.655 [graph_manager.cc:1135][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [68] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.701 [graph_manager.cc:2975][EVENT]167203 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [27] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.733 [graph_manager.cc:2981][EVENT]167203 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.747 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.757 [graph_manager.cc:2986][EVENT]167203 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.434.766 [graph_manager.cc:1136][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [93] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.435.106 [graph_manager.cc:3555][EVENT]167203 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [299] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.435.235 [engine_partitioner.cc:1139][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.435.264 [engine_partitioner.cc:1142][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.435.414 [engine_partitioner.cc:1148][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [139] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.435.450 [engine_partitioner.cc:1155][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.435.498 [engine_partitioner.cc:1164][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.435.532 [graph_builder.cc:865][EVENT]167203 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [358] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:33.436.080 [logger.cc:1071] 167203 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:37:33.436.116 [task_generator.cc:804][EVENT]167203 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [191] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.436.204 [task_generator.cc:805][EVENT]167203 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [75] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.437.821 [task_generator.cc:814][EVENT]167203 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1602] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.437.838 [task_generator.cc:954][EVENT]167203 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1914] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.437.911 [task_generator.cc:967][EVENT]167203 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [40] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:33.437.938 [logger.cc:1084] 167203 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:37:33.441.254 [graph_manager.cc:1152][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [6453] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.441.296 [graph_manager.cc:1164][EVENT]167203 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:33.441.337 [graph_manager.cc:1271][EVENT]167203 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [171982] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.441.348 [graph_manager.cc:1272][EVENT]167203 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:37:33.441.786 [atrace_api.c:93](tid:167203) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:37:33.441.808 [atrace_api.c:95](tid:167203) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:37:33.469.423 [graph_converter.cc:838][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [11205] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.469.657 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of ZeroCopy is [173] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.471.274 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CEM is [1592] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.471.682 [copy_flow_launch_fuse.cc:395][EVENT]167203 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [377] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.471.707 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [405] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.471.981 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [261] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.472.063 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [61] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.472.131 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of ZeroCopy is [51] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.472.585 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CEM is [439] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.472.805 [copy_flow_launch_fuse.cc:395][EVENT]167203 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [195] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.472.839 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [231] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.472.908 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [58] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.472.967 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [48] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.473.026 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of ZeroCopy is [47] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.473.243 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CEM is [206] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.473.435 [copy_flow_launch_fuse.cc:395][EVENT]167203 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [175] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.473.451 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [193] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.473.515 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [53] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.473.572 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [46] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.473.589 [graph_converter.cc:849][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4112] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.474.328 [graph_converter.cc:853][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [728] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.476.311 [graph_converter.cc:857][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1959] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.476.696 [graph_converter.cc:862][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [357] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.556.053 [graph_var_manager.cc:1424][EVENT]167202 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:33.556.141 [graph_manager.cc:1248][EVENT]167202 PreRun:PreRun start: graph node size 4, session id 8, graph id 7, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:33.556.761 [atrace_api.c:28](tid:167202) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:33.556.819 [trace_rb_log.c:84](tid:167202) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:33.556.832 [atrace_api.c:32](tid:167202) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:33.556.853 [client_manager.cpp:157][SetProfilingCallback][tid:167202] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:33.557.281 [parallel_partitioner.cc:165][EVENT]167202 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.557.317 [parallel_partitioner.cc:178][EVENT]167202 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.557.364 [graph_prepare.cc:1378][EVENT]167202 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.557.521 [graph_manager.cc:1050][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [174] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.557.561 [graph_manager.cc:1052][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.557.740 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.557.771 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.557.821 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.557.834 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.557.881 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.557.894 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.557.912 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.558.002 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.558.022 [graph_manager.cc:1054][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [449] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.558.263 [graph_manager.cc:1055][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [228] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.559.348 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:33.559.374 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.559.385 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.559.395 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferShapePass is [339] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.559.404 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [15] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.559.413 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:33.559.422 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [69] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.559.430 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.559.438 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferValuePass is [7] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.561.217 [graph_manager.cc:1056][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2934] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.561.284 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.561.311 [graph_prepare.cc:1982][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [62] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.561.831 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:33.561.855 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.561.866 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.561.876 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferShapePass is [265] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.561.885 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.561.893 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:33.561.902 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.561.910 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.561.919 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.561.977 [graph_prepare.cc:1983][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [652] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.562.002 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.562.014 [graph_prepare.cc:1984][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.562.029 [graph_prepare.cc:1985][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.562.044 [graph_prepare.cc:1986][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.562.055 [graph_prepare.cc:1987][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.562.070 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.562.081 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.562.095 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.562.187 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.562.198 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.562.207 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.562.216 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.562.231 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.562.241 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.562.249 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.562.258 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.562.267 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of StopGradientPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.562.276 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.562.284 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.562.293 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.562.301 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.562.309 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [7] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.562.317 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.562.326 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.562.349 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.562.364 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.562.397 [graph_prepare.cc:1988][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [333] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.562.410 [graph_manager.cc:1065][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1163] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.574.637 [graph_manager.cc:1077][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12208] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.574.764 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.574.814 [graph_manager.cc:1080][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [141] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.117 [graph_manager.cc:1081][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [6287] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.172 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.188 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.213 [graph_manager.cc:1082][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [51] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.247 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.263 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.277 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.310 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [24] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.327 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.341 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.355 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.396 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [33] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.418 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.437 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.468 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.487 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.500 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.512 [graph_manager.cc:2700][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [272] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.637 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.581.654 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.581.666 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.581.679 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.581.708 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.581.722 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.581.732 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.581.740 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.581.758 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.581.767 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.581.779 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.581.788 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.581.799 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.581.808 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.581.816 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.581.826 [graph_manager.cc:2741][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [295] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.835 [graph_manager.cc:2752][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.857 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.868 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.886 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.901 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.912 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.923 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.941 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.955 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.967 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.977 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.581.990 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.582.001 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.582.019 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.582.041 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.582.051 [graph_manager.cc:2810][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [198] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.582.082 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.582.094 [graph_manager.cc:2821][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [34] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.582.123 [graph_manager.cc:1087][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [891] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.582.256 [graph_manager.cc:1088][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [120] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.582.297 [graph_manager.cc:1089][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.582.317 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.582.333 [graph_manager.cc:1097][EVENT]167202 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:33.582.355 [graph_manager.cc:3325][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.582.734 [engine_place.cc:144][EVENT]167202 Run:The time cost of AIcoreEngine::CheckSupported is [273] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.582.758 [engine_place.cc:144][EVENT]167202 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.582.768 [engine_place.cc:144][EVENT]167202 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.582.849 [graph_manager.cc:3351][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [481] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.582.870 [graph_manager.cc:3364][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.582.943 [engine_partitioner.cc:1139][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.582.961 [engine_partitioner.cc:1142][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.583.123 [engine_partitioner.cc:1148][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [152] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.583.172 [engine_partitioner.cc:1155][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.583.224 [engine_partitioner.cc:1164][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [40] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.583.259 [graph_manager.cc:3405][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [377] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.583.287 [graph_manager.cc:3412][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.624.728 [graph_manager.cc:3422][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [41425] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.624.787 [graph_manager.cc:3428][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.624.967 [graph_manager.cc:3467][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [157] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.624.985 [graph_manager.cc:3377][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [42104] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.003 [graph_manager.cc:1106][EVENT]167202 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [42654] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.015 [graph_manager.cc:1115][EVENT]167202 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:33.625.041 [graph_manager.cc:1130][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.076 [graph_manager.cc:1131][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.104 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.122 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.132 [graph_manager.cc:2837][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.247 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.625.260 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.625.270 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.625.278 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.625.287 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.625.295 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [8] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.625.306 [graph_manager.cc:2864][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [155] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.318 [graph_manager.cc:2872][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.337 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.362 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.378 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.392 [compile_nodes_pass.cc:88][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.401 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.411 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.495 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [74] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.526 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.539 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.552 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.567 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.576 [graph_manager.cc:2927][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [243] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.588 [graph_manager.cc:2937][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.602 [graph_manager.cc:2943][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.613 [graph_manager.cc:2950][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.845 [graph_manager.cc:2958][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.879 [graph_manager.cc:1132][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [789] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.625.968 [graph_manager.cc:1135][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [75] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.626.004 [graph_manager.cc:2975][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.626.037 [graph_manager.cc:2981][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.626.051 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.626.061 [graph_manager.cc:2986][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.626.078 [graph_manager.cc:1136][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [94] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.626.215 [graph_manager.cc:3555][EVENT]167202 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [102] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.626.320 [engine_partitioner.cc:1139][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.626.339 [engine_partitioner.cc:1142][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.626.490 [engine_partitioner.cc:1148][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [141] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.626.529 [engine_partitioner.cc:1155][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [24] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.626.577 [engine_partitioner.cc:1164][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.626.604 [graph_builder.cc:865][EVENT]167202 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [324] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:33.626.935 [logger.cc:1071] 167202 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:37:33.626.968 [task_generator.cc:804][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [95] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.627.032 [task_generator.cc:805][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [52] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.627.729 [task_generator.cc:814][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [681] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.627.747 [task_generator.cc:954][EVENT]167202 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [874] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.627.810 [task_generator.cc:967][EVENT]167202 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [36] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:33.627.830 [logger.cc:1084] 167202 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:37:33.628.418 [graph_manager.cc:1152][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2314] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.628.450 [graph_manager.cc:1164][EVENT]167202 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:33.628.493 [graph_manager.cc:1271][EVENT]167202 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [71309] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.628.504 [graph_manager.cc:1272][EVENT]167202 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:37:33.628.829 [atrace_api.c:93](tid:167202) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:37:33.628.847 [atrace_api.c:95](tid:167202) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:37:33.641.764 [graph_converter.cc:838][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [4022] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.641.943 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [128] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.642.465 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [489] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.642.682 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [193] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.642.702 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [215] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.642.924 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [209] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.642.964 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.642.995 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.643.188 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [181] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.643.274 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [68] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.643.288 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [83] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.643.318 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.643.343 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.643.369 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.643.444 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [66] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.643.512 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [57] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.643.524 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [69] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.643.551 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.643.577 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.643.591 [graph_converter.cc:849][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1782] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.643.817 [graph_converter.cc:853][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [216] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.644.587 [graph_converter.cc:857][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [755] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.644.735 [graph_converter.cc:862][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [126] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.711.893 [graph_var_manager.cc:1424][EVENT]167201 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:33.711.982 [graph_manager.cc:1248][EVENT]167201 PreRun:PreRun start: graph node size 3, session id 9, graph id 8, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:33.712.605 [atrace_api.c:28](tid:167201) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:33.712.661 [trace_rb_log.c:84](tid:167201) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:33.712.674 [atrace_api.c:32](tid:167201) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:33.712.694 [client_manager.cpp:157][SetProfilingCallback][tid:167201] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:33.713.401 [parallel_partitioner.cc:165][EVENT]167201 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.713.438 [parallel_partitioner.cc:178][EVENT]167201 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.713.485 [graph_prepare.cc:1378][EVENT]167201 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.713.897 [graph_manager.cc:1050][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [429] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.713.924 [graph_manager.cc:1052][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.714.051 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.714.080 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.714.130 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.714.143 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.714.191 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.714.204 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.714.221 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.714.305 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.714.325 [graph_manager.cc:1054][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [390] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.714.569 [graph_manager.cc:1055][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [230] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.715.388 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:33.715.414 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.715.426 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.715.435 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [248] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.715.458 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.715.467 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [4] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:33.715.476 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.715.485 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.715.493 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.101 [graph_manager.cc:1056][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2510] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.717.165 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.184 [graph_prepare.cc:1982][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [50] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.717.515 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:33.717.537 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.548 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.557 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [173] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.566 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.575 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:33.717.583 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.591 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.600 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.624 [graph_prepare.cc:1983][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [427] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.717.646 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.717.657 [graph_prepare.cc:1984][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.717.670 [graph_prepare.cc:1985][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.717.684 [graph_prepare.cc:1986][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.717.712 [graph_prepare.cc:1987][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.717.736 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.717.748 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.717.762 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.717.842 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.854 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.863 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.871 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.880 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.888 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.897 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.905 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.913 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.921 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [0] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.929 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.938 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.946 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.954 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.962 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.970 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.717.992 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.718.005 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.718.035 [graph_prepare.cc:1988][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [304] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.718.048 [graph_manager.cc:1065][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [914] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.729.778 [graph_manager.cc:1077][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11705] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.729.843 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.729.889 [graph_manager.cc:1080][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [78] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.732.938 [graph_manager.cc:1081][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3033] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.732.977 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.732.992 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.004 [graph_manager.cc:1082][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.034 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.049 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.063 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.093 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.108 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.122 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.135 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.173 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [28] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.193 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.211 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.235 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.249 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.260 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.269 [graph_manager.cc:2700][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [239] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.386 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.733.399 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.733.409 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.733.418 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.733.427 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.733.435 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.733.444 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.733.452 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.733.460 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.733.468 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.733.477 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [6] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.733.485 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.733.493 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.733.502 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.733.510 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.733.519 [graph_manager.cc:2741][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [220] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.528 [graph_manager.cc:2752][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.550 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.561 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.577 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.591 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.602 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.614 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.637 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.652 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.664 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.675 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.696 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.709 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.727 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.740 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.749 [graph_manager.cc:2810][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [202] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.775 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.733.786 [graph_manager.cc:2821][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [28] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.814 [graph_manager.cc:1087][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [792] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.943 [graph_manager.cc:1088][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [115] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.980 [graph_manager.cc:1089][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.733.997 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.734.011 [graph_manager.cc:1097][EVENT]167201 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:33.734.032 [graph_manager.cc:3325][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.734.381 [engine_place.cc:144][EVENT]167201 Run:The time cost of AIcoreEngine::CheckSupported is [258] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.734.404 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.734.414 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.734.487 [graph_manager.cc:3351][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [442] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.734.505 [graph_manager.cc:3364][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.734.580 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.734.596 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.734.727 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [121] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.734.769 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [28] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.734.813 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [34] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.734.845 [graph_manager.cc:3405][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [320] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.734.864 [graph_manager.cc:3412][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.742.861 [graph_manager.cc:3422][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [7982] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.742.896 [graph_manager.cc:3428][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.020 [graph_manager.cc:3467][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [103] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.037 [graph_manager.cc:3377][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [8513] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.052 [graph_manager.cc:1106][EVENT]167201 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [9026] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.064 [graph_manager.cc:1115][EVENT]167201 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:33.743.087 [graph_manager.cc:1130][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.118 [graph_manager.cc:1131][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.142 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.158 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.168 [graph_manager.cc:2837][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.243 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.743.255 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.743.273 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.743.282 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.743.291 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [4] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.743.300 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:33.743.309 [graph_manager.cc:2864][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [125] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.322 [graph_manager.cc:2872][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.340 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.353 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.368 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.381 [compile_nodes_pass.cc:88][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.390 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.400 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.471 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [62] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.496 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.510 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.522 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.535 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.544 [graph_manager.cc:2927][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [207] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.556 [graph_manager.cc:2937][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.569 [graph_manager.cc:2943][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.580 [graph_manager.cc:2950][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.761 [graph_manager.cc:2958][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [34] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.791 [graph_manager.cc:1132][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [661] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.859 [graph_manager.cc:1135][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [54] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.891 [graph_manager.cc:2975][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.927 [graph_manager.cc:2981][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.942 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.952 [graph_manager.cc:2986][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.743.962 [graph_manager.cc:1136][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [88] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.744.075 [graph_manager.cc:3555][EVENT]167201 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [83] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.744.159 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.744.174 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.744.271 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [87] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.744.299 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.744.336 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [27] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.744.357 [graph_builder.cc:865][EVENT]167201 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [230] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:33.744.650 [logger.cc:1071] 167201 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:37:33.744.680 [task_generator.cc:804][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [83] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.744.738 [task_generator.cc:805][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [46] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.745.351 [task_generator.cc:814][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [599] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.745.364 [task_generator.cc:954][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [768] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.745.425 [task_generator.cc:967][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [35] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:33.745.458 [logger.cc:1084] 167201 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:37:33.745.628 [graph_manager.cc:1152][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [1643] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.745.646 [graph_manager.cc:1164][EVENT]167201 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:33.745.681 [graph_manager.cc:1271][EVENT]167201 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [32372] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.745.709 [graph_manager.cc:1272][EVENT]167201 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:37:33.746.022 [atrace_api.c:93](tid:167201) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:37:33.746.038 [atrace_api.c:95](tid:167201) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:37:33.750.628 [graph_converter.cc:838][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1320] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.750.789 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [118] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.751.223 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [413] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.751.403 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [158] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.751.421 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [178] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.751.632 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [199] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.751.670 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.751.700 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.751.877 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [165] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.751.955 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [61] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.751.968 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [74] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.751.995 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.752.018 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.752.042 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.752.109 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [58] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.752.172 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [52] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.752.183 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [63] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.752.207 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.752.229 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.752.250 [graph_converter.cc:849][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1586] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.752.456 [graph_converter.cc:853][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [196] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.753.095 [graph_converter.cc:857][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [624] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.753.223 [graph_converter.cc:862][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [107] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.822.733 [graph_var_manager.cc:1424][EVENT]167203 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:33.822.821 [graph_manager.cc:1248][EVENT]167203 PreRun:PreRun start: graph node size 4, session id 10, graph id 9, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:33.823.264 [atrace_api.c:28](tid:167203) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:33.823.324 [trace_rb_log.c:84](tid:167203) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:33.823.340 [atrace_api.c:32](tid:167203) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:33.823.359 [client_manager.cpp:157][SetProfilingCallback][tid:167203] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:33.823.839 [parallel_partitioner.cc:165][EVENT]167203 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.823.880 [parallel_partitioner.cc:178][EVENT]167203 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.823.929 [graph_prepare.cc:1378][EVENT]167203 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.824.103 [graph_manager.cc:1050][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [191] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.824.130 [graph_manager.cc:1052][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.824.278 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.824.309 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.824.361 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.824.377 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.824.422 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.824.435 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.824.452 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.824.562 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.824.583 [graph_manager.cc:1054][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [440] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.824.821 [graph_manager.cc:1055][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [226] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.825.984 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:33.826.013 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.826.025 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.826.035 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferShapePass is [471] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.826.044 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.826.053 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [4] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:33.826.061 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.826.070 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.826.078 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.828.624 [graph_manager.cc:1056][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3780] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.828.692 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.828.711 [graph_prepare.cc:1982][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [54] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.829.292 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:33.829.319 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.330 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.340 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferShapePass is [365] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.349 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.358 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:33.829.366 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.375 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.394 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.422 [graph_prepare.cc:1983][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [697] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.829.445 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.829.457 [graph_prepare.cc:1984][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.829.471 [graph_prepare.cc:1985][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.829.487 [graph_prepare.cc:1986][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.829.497 [graph_prepare.cc:1987][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.829.511 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.829.522 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.829.535 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.829.624 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.636 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.645 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.654 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.662 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.671 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.679 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.705 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.714 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.722 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.731 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.739 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.753 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.762 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.770 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.779 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:33.829.803 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.829.815 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.829.848 [graph_prepare.cc:1988][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [343] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.829.861 [graph_manager.cc:1065][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1204] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.842.854 [graph_manager.cc:1077][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12973] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.842.928 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.842.976 [graph_manager.cc:1080][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [85] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.853.432 [graph_manager.cc:1081][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [10439] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.853.488 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.853.506 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.853.518 [graph_manager.cc:1082][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.853.551 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.853.565 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.853.580 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.853.756 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [166] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.853.775 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.853.867 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [82] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.853.884 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.853.953 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [41] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.853.975 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.853.996 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.084 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [77] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.101 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.115 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.125 [graph_manager.cc:2700][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [580] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.363 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.854.378 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AddNPass is [3] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.854.388 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.854.397 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.854.406 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.854.415 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CastRemovePass is [40] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.854.424 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [5] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.854.432 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.854.441 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [13] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.854.449 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.854.458 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [17] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.854.466 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.854.474 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.854.483 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [12] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.854.491 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.854.520 [graph_manager.cc:2741][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [377] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.531 [graph_manager.cc:2752][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.556 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.567 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.588 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.604 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.616 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.629 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.649 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.662 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.675 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.685 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.697 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.709 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.730 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.743 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.752 [graph_manager.cc:2810][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [201] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.796 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.854.807 [graph_manager.cc:2821][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [46] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.854.836 [graph_manager.cc:1087][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1299] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.855.408 [graph_manager.cc:1088][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [557] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.855.469 [graph_manager.cc:1089][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [32] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.855.491 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.855.516 [graph_manager.cc:1097][EVENT]167203 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:33.855.540 [graph_manager.cc:3325][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.865.789 [engine_place.cc:144][EVENT]167203 Run:The time cost of AIcoreEngine::CheckSupported is [10037] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.865.821 [engine_place.cc:144][EVENT]167203 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.865.832 [engine_place.cc:144][EVENT]167203 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.865.928 [graph_manager.cc:3351][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10374] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.865.949 [graph_manager.cc:3364][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.866.029 [engine_partitioner.cc:1139][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [29] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.866.061 [engine_partitioner.cc:1142][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.866.236 [engine_partitioner.cc:1148][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [165] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.866.278 [engine_partitioner.cc:1155][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [29] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.866.327 [engine_partitioner.cc:1164][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.866.363 [graph_manager.cc:3405][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [401] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.866.381 [graph_manager.cc:3412][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.979.793 [graph_manager.cc:3422][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [113395] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.979.850 [graph_manager.cc:3428][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.044 [graph_manager.cc:3467][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [170] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.066 [graph_manager.cc:3377][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [114106] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.085 [graph_manager.cc:1106][EVENT]167203 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [124553] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.100 [graph_manager.cc:1115][EVENT]167203 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:33.980.143 [graph_manager.cc:1130][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.180 [graph_manager.cc:1131][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.210 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.231 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.245 [graph_manager.cc:2837][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [48] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.392 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [29] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.980.408 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.980.418 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.980.427 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of BitcastPass is [3] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.980.436 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [9] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.980.444 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [18] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:33.980.455 [graph_manager.cc:2864][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [192] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.468 [graph_manager.cc:2872][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.486 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.501 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.519 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.533 [compile_nodes_pass.cc:88][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.546 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.559 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.669 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [100] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.723 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.745 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.760 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.775 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.784 [graph_manager.cc:2927][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [302] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.797 [graph_manager.cc:2937][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.813 [graph_manager.cc:2943][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.980.824 [graph_manager.cc:2950][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.981.048 [graph_manager.cc:2958][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [55] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.981.084 [graph_manager.cc:1132][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [890] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.981.168 [graph_manager.cc:1135][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [71] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.981.208 [graph_manager.cc:2975][EVENT]167203 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.981.239 [graph_manager.cc:2981][EVENT]167203 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.981.253 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.981.263 [graph_manager.cc:2986][EVENT]167203 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.981.272 [graph_manager.cc:1136][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [86] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.981.607 [graph_manager.cc:3555][EVENT]167203 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [293] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.981.758 [engine_partitioner.cc:1139][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [33] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.981.792 [engine_partitioner.cc:1142][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.981.947 [engine_partitioner.cc:1148][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [142] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.981.986 [engine_partitioner.cc:1155][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.982.030 [engine_partitioner.cc:1164][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [33] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.982.065 [graph_builder.cc:865][EVENT]167203 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [384] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:33.982.461 [logger.cc:1071] 167203 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:37:33.982.495 [task_generator.cc:804][EVENT]167203 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [88] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.982.584 [task_generator.cc:805][EVENT]167203 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [72] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.984.253 [task_generator.cc:814][EVENT]167203 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1653] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.984.271 [task_generator.cc:954][EVENT]167203 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1864] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.984.345 [task_generator.cc:967][EVENT]167203 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [40] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:33.984.371 [logger.cc:1084] 167203 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:37:33.985.796 [graph_manager.cc:1152][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4489] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.985.840 [graph_manager.cc:1164][EVENT]167203 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:33.985.879 [graph_manager.cc:1271][EVENT]167203 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [162133] micro second. [INFO] GE(164039,python):2024-01-10-11:37:33.985.890 [graph_manager.cc:1272][EVENT]167203 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:37:33.986.218 [atrace_api.c:93](tid:167203) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:37:33.986.236 [atrace_api.c:95](tid:167203) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:37:34.015.597 [graph_converter.cc:838][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [10907] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.015.828 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of ZeroCopy is [169] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.017.310 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CEM is [1457] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.017.729 [copy_flow_launch_fuse.cc:395][EVENT]167203 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [390] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.017.755 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [418] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.018.023 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [254] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.018.106 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [62] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.018.174 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of ZeroCopy is [50] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.018.627 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CEM is [438] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.018.841 [copy_flow_launch_fuse.cc:395][EVENT]167203 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [188] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.018.862 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [210] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.018.944 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [55] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.019.003 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [45] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.019.060 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of ZeroCopy is [46] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.019.280 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CEM is [208] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.019.469 [copy_flow_launch_fuse.cc:395][EVENT]167203 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [174] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.019.485 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [192] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.019.546 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [50] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.019.602 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [44] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.019.620 [graph_converter.cc:849][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [3968] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.020.324 [graph_converter.cc:853][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [692] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.022.297 [graph_converter.cc:857][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1951] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.022.682 [graph_converter.cc:862][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [355] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.096.475 [graph_var_manager.cc:1424][EVENT]167203 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:34.096.564 [graph_manager.cc:1248][EVENT]167203 PreRun:PreRun start: graph node size 4, session id 11, graph id 10, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:34.096.830 [atrace_api.c:28](tid:167203) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:34.096.865 [trace_rb_log.c:84](tid:167203) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:34.096.881 [atrace_api.c:32](tid:167203) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:34.096.902 [client_manager.cpp:157][SetProfilingCallback][tid:167203] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:34.097.327 [parallel_partitioner.cc:165][EVENT]167203 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.097.368 [parallel_partitioner.cc:178][EVENT]167203 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.097.416 [graph_prepare.cc:1378][EVENT]167203 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.097.612 [graph_manager.cc:1050][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [213] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.097.638 [graph_manager.cc:1052][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.097.812 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.097.848 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.097.902 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [41] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.097.918 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.097.963 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.097.976 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.097.994 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.098.081 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.098.102 [graph_manager.cc:1054][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [435] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.098.345 [graph_manager.cc:1055][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [227] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.099.491 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:34.099.519 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.099.531 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.099.540 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferShapePass is [352] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.099.549 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.099.558 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:34.099.566 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [75] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.099.574 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.099.583 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.101.431 [graph_manager.cc:1056][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3063] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.101.498 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.101.516 [graph_prepare.cc:1982][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.102.000 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:34.102.025 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.036 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.046 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferShapePass is [274] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.055 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.063 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:34.102.072 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.081 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.089 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.115 [graph_prepare.cc:1983][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [576] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.102.139 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.102.151 [graph_prepare.cc:1984][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.102.164 [graph_prepare.cc:1985][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.102.178 [graph_prepare.cc:1986][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.102.189 [graph_prepare.cc:1987][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.102.204 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.102.215 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.102.228 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.102.320 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.331 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.340 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.348 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.357 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DropOutPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.372 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.381 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.389 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.398 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.406 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.414 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.422 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.430 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.439 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.447 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.455 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.102.479 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.102.493 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.102.527 [graph_prepare.cc:1988][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [329] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.102.539 [graph_manager.cc:1065][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1077] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.114.533 [graph_manager.cc:1077][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11973] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.114.607 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.114.661 [graph_manager.cc:1080][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [92] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.118.870 [graph_manager.cc:1081][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [4194] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.118.915 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.118.930 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.118.942 [graph_manager.cc:1082][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.118.988 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.002 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.017 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.051 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [24] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.065 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.080 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.092 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.134 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [32] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.152 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.171 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.199 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.216 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.228 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.237 [graph_manager.cc:2700][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [253] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.356 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.119.370 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.119.380 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.119.388 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.119.397 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.119.405 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CastRemovePass is [7] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.119.414 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.119.422 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.119.437 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.119.446 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.119.454 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.119.463 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.119.471 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.119.479 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.119.487 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.119.497 [graph_manager.cc:2741][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [243] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.506 [graph_manager.cc:2752][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.528 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.540 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.556 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.570 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.581 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.594 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.612 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.626 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.638 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.648 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.661 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.672 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.691 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.704 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.718 [graph_manager.cc:2810][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [193] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.748 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.119.759 [graph_manager.cc:2821][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [33] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.789 [graph_manager.cc:1087][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [813] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.922 [graph_manager.cc:1088][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [119] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.964 [graph_manager.cc:1089][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.982 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.119.997 [graph_manager.cc:1097][EVENT]167203 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:34.120.019 [graph_manager.cc:3325][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.121.042 [engine_place.cc:144][EVENT]167203 Run:The time cost of AIcoreEngine::CheckSupported is [916] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.121.068 [engine_place.cc:144][EVENT]167203 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.121.078 [engine_place.cc:144][EVENT]167203 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.121.160 [graph_manager.cc:3351][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [1127] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.121.179 [graph_manager.cc:3364][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.121.243 [engine_partitioner.cc:1139][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.121.261 [engine_partitioner.cc:1142][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.121.448 [engine_partitioner.cc:1148][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [177] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.121.492 [engine_partitioner.cc:1155][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.121.542 [engine_partitioner.cc:1164][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.121.575 [graph_manager.cc:3405][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [382] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.121.594 [graph_manager.cc:3412][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.248 [graph_manager.cc:3422][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [20631] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.296 [graph_manager.cc:3428][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.449 [graph_manager.cc:3467][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [134] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.469 [graph_manager.cc:3377][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [21277] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.486 [graph_manager.cc:1106][EVENT]167203 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [22473] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.499 [graph_manager.cc:1115][EVENT]167203 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:34.142.523 [graph_manager.cc:1130][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.559 [graph_manager.cc:1131][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.584 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.601 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.611 [graph_manager.cc:2837][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.704 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.142.717 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.142.727 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CondRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.142.735 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.142.744 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.142.753 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.142.763 [graph_manager.cc:2864][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [135] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.775 [graph_manager.cc:2872][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.795 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.809 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.834 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.848 [compile_nodes_pass.cc:88][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.858 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.867 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.956 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [80] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.985 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.142.998 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.012 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.026 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.035 [graph_manager.cc:2927][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [244] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.047 [graph_manager.cc:2937][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.062 [graph_manager.cc:2943][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.072 [graph_manager.cc:2950][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.268 [graph_manager.cc:2958][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [41] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.300 [graph_manager.cc:1132][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [727] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.377 [graph_manager.cc:1135][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [64] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.411 [graph_manager.cc:2975][EVENT]167203 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.444 [graph_manager.cc:2981][EVENT]167203 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.459 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.469 [graph_manager.cc:2986][EVENT]167203 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.486 [graph_manager.cc:1136][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [92] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.613 [graph_manager.cc:3555][EVENT]167203 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [92] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.708 [engine_partitioner.cc:1139][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.726 [engine_partitioner.cc:1142][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.867 [engine_partitioner.cc:1148][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [131] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.901 [engine_partitioner.cc:1155][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.943 [engine_partitioner.cc:1164][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.143.967 [graph_builder.cc:865][EVENT]167203 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [295] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:34.144.276 [logger.cc:1071] 167203 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:37:34.144.307 [task_generator.cc:804][EVENT]167203 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [86] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.144.371 [task_generator.cc:805][EVENT]167203 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [52] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.145.066 [task_generator.cc:814][EVENT]167203 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [680] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.145.080 [task_generator.cc:954][EVENT]167203 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [859] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.145.140 [task_generator.cc:967][EVENT]167203 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [34] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:34.145.158 [logger.cc:1084] 167203 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:37:34.146.560 [graph_manager.cc:1152][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3045] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.146.598 [graph_manager.cc:1164][EVENT]167203 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:34.146.636 [graph_manager.cc:1271][EVENT]167203 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [49405] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.146.646 [graph_manager.cc:1272][EVENT]167203 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:37:34.146.967 [atrace_api.c:93](tid:167203) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:37:34.146.984 [atrace_api.c:95](tid:167203) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:37:34.169.906 [graph_converter.cc:838][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [7530] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.170.119 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of ZeroCopy is [158] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.170.688 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CEM is [531] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.170.909 [copy_flow_launch_fuse.cc:395][EVENT]167203 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [196] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.170.930 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [218] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.171.232 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [291] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.171.276 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [25] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.171.310 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of ZeroCopy is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.171.519 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CEM is [198] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.171.609 [copy_flow_launch_fuse.cc:395][EVENT]167203 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [72] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.171.624 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [86] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.171.655 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.171.681 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.171.709 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of ZeroCopy is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.171.788 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CEM is [69] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.171.859 [copy_flow_launch_fuse.cc:395][EVENT]167203 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [61] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.171.871 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [73] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.171.899 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.171.923 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.171.936 [graph_converter.cc:849][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1981] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.172.183 [graph_converter.cc:853][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [237] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.172.974 [graph_converter.cc:857][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [775] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.173.130 [graph_converter.cc:862][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [132] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.247.131 [graph_var_manager.cc:1424][EVENT]167204 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:34.247.216 [graph_manager.cc:1248][EVENT]167204 PreRun:PreRun start: graph node size 4, session id 12, graph id 11, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:34.247.471 [atrace_api.c:28](tid:167204) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:34.247.522 [trace_rb_log.c:84](tid:167204) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:34.247.536 [atrace_api.c:32](tid:167204) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:34.247.556 [client_manager.cpp:157][SetProfilingCallback][tid:167204] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:34.247.975 [parallel_partitioner.cc:165][EVENT]167204 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.248.011 [parallel_partitioner.cc:178][EVENT]167204 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.248.059 [graph_prepare.cc:1378][EVENT]167204 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.248.229 [graph_manager.cc:1050][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [188] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.248.255 [graph_manager.cc:1052][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.248.395 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.248.424 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.248.470 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.248.483 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.248.529 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.248.543 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.248.561 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.248.648 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.248.668 [graph_manager.cc:1054][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [402] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.248.905 [graph_manager.cc:1055][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [224] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.249.977 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:34.250.005 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.250.017 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.250.026 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferShapePass is [375] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.250.035 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.250.052 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:34.250.062 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [27] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.250.071 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.250.079 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferValuePass is [11] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.251.843 [graph_manager.cc:1056][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2917] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.251.908 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.251.927 [graph_prepare.cc:1982][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [52] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.252.409 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:34.252.431 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.442 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.451 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferShapePass is [277] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.460 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.469 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:34.252.477 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.486 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.494 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.520 [graph_prepare.cc:1983][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [579] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.252.542 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.252.553 [graph_prepare.cc:1984][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.252.566 [graph_prepare.cc:1985][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.252.580 [graph_prepare.cc:1986][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.252.591 [graph_prepare.cc:1987][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.252.613 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.252.625 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.252.639 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.252.729 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.741 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.749 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.758 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.766 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DropOutPass is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.775 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.783 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.792 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.800 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.808 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.817 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.825 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SnapshotPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.833 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.841 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.849 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.857 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.252.880 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.252.893 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.252.925 [graph_prepare.cc:1988][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [318] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.252.939 [graph_manager.cc:1065][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1065] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.265.232 [graph_manager.cc:1077][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12263] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.265.306 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.265.355 [graph_manager.cc:1080][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [86] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.273.226 [graph_manager.cc:1081][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [7853] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.273.279 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.273.294 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.273.305 [graph_manager.cc:1082][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.273.336 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.273.349 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.273.363 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.273.460 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [87] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.273.478 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.273.524 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.273.538 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.273.579 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [30] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.273.599 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.273.618 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.273.645 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.273.660 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.273.672 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.273.681 [graph_manager.cc:2700][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [349] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.273.837 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.273.851 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.273.860 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.273.869 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.273.878 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.273.887 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CastRemovePass is [9] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.273.896 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.273.905 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.273.913 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.273.922 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.273.930 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.273.938 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.273.946 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.273.954 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.273.963 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.273.973 [graph_manager.cc:2741][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [245] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.273.982 [graph_manager.cc:2752][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.004 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.015 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.032 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.047 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.059 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.071 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.097 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.111 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.123 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.133 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.146 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.157 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.176 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.189 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.199 [graph_manager.cc:2810][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [199] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.228 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.274.239 [graph_manager.cc:2821][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.267 [graph_manager.cc:1087][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [942] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.401 [graph_manager.cc:1088][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [122] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.442 [graph_manager.cc:1089][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.459 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.474 [graph_manager.cc:1097][EVENT]167204 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:34.274.495 [graph_manager.cc:3325][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.883 [engine_place.cc:144][EVENT]167204 Run:The time cost of AIcoreEngine::CheckSupported is [284] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.907 [engine_place.cc:144][EVENT]167204 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.917 [engine_place.cc:144][EVENT]167204 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.274.995 [graph_manager.cc:3351][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [486] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.275.012 [graph_manager.cc:3364][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.275.090 [engine_partitioner.cc:1139][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.275.108 [engine_partitioner.cc:1142][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.275.272 [engine_partitioner.cc:1148][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [154] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.275.316 [engine_partitioner.cc:1155][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.275.363 [engine_partitioner.cc:1164][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.275.397 [graph_manager.cc:3405][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [364] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.275.415 [graph_manager.cc:3412][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.292.664 [graph_manager.cc:3422][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [17235] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.292.706 [graph_manager.cc:3428][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.292.851 [graph_manager.cc:3467][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [124] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.292.870 [graph_manager.cc:3377][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [17838] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.292.886 [graph_manager.cc:1106][EVENT]167204 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [18397] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.292.898 [graph_manager.cc:1115][EVENT]167204 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:34.292.922 [graph_manager.cc:1130][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.292.953 [graph_manager.cc:1131][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.292.978 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.292.995 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.005 [graph_manager.cc:2837][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.095 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.293.107 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.293.125 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.293.135 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.293.143 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.293.152 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [10] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.293.161 [graph_manager.cc:2864][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [140] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.173 [graph_manager.cc:2872][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.193 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.206 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.221 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.234 [compile_nodes_pass.cc:88][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.244 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.253 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.336 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [74] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.363 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.375 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.389 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.402 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.411 [graph_manager.cc:2927][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [222] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.423 [graph_manager.cc:2937][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.437 [graph_manager.cc:2943][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.448 [graph_manager.cc:2950][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.635 [graph_manager.cc:2958][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.673 [graph_manager.cc:1132][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [706] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.757 [graph_manager.cc:1135][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [58] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.792 [graph_manager.cc:2975][EVENT]167204 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.823 [graph_manager.cc:2981][EVENT]167204 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.837 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.846 [graph_manager.cc:2986][EVENT]167204 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.855 [graph_manager.cc:1136][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [81] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.293.994 [graph_manager.cc:3555][EVENT]167204 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [106] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.294.088 [engine_partitioner.cc:1139][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.294.104 [engine_partitioner.cc:1142][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.294.226 [engine_partitioner.cc:1148][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [112] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.294.259 [engine_partitioner.cc:1155][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.294.300 [engine_partitioner.cc:1164][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [29] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.294.322 [graph_builder.cc:865][EVENT]167204 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [270] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:34.294.619 [logger.cc:1071] 167204 ModelBindStream: model_id=1856, stream_id=65, flag=0. [INFO] GE(164039,python):2024-01-10-11:37:34.294.648 [task_generator.cc:804][EVENT]167204 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [81] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.294.708 [task_generator.cc:805][EVENT]167204 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [48] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.295.359 [task_generator.cc:814][EVENT]167204 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [637] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.295.373 [task_generator.cc:954][EVENT]167204 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [806] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.295.433 [task_generator.cc:967][EVENT]167204 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [34] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:34.295.452 [logger.cc:1084] 167204 ModelUnbindStream: model_id=1856, stream_id=65, [INFO] GE(164039,python):2024-01-10-11:37:34.296.056 [graph_manager.cc:1152][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2174] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.296.085 [graph_manager.cc:1164][EVENT]167204 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:34.296.121 [graph_manager.cc:1271][EVENT]167204 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [48236] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.296.132 [graph_manager.cc:1272][EVENT]167204 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:37:34.296.449 [atrace_api.c:93](tid:167204) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:37:34.296.467 [atrace_api.c:95](tid:167204) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:37:34.308.084 [graph_converter.cc:838][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [3883] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.308.250 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of ZeroCopy is [121] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.308.741 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CEM is [469] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.308.952 [copy_flow_launch_fuse.cc:395][EVENT]167204 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [188] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.308.971 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [209] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.309.191 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [207] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.309.229 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.309.259 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.309.451 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CEM is [181] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.309.534 [copy_flow_launch_fuse.cc:395][EVENT]167204 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [66] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.309.547 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [80] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.309.576 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.309.601 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.309.626 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.309.730 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CEM is [94] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.309.802 [copy_flow_launch_fuse.cc:395][EVENT]167204 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [57] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.309.813 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [68] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.309.840 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.309.863 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.309.887 [graph_converter.cc:849][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1763] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.310.111 [graph_converter.cc:853][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [214] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.310.845 [graph_converter.cc:857][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [720] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.310.989 [graph_converter.cc:862][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [121] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.374.644 [graph_var_manager.cc:1424][EVENT]167201 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:34.374.734 [graph_manager.cc:1248][EVENT]167201 PreRun:PreRun start: graph node size 4, session id 13, graph id 12, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:34.375.004 [atrace_api.c:28](tid:167201) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:34.375.039 [trace_rb_log.c:84](tid:167201) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:34.375.053 [atrace_api.c:32](tid:167201) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:34.375.072 [client_manager.cpp:157][SetProfilingCallback][tid:167201] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:34.375.485 [parallel_partitioner.cc:165][EVENT]167201 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.375.521 [parallel_partitioner.cc:178][EVENT]167201 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.375.569 [graph_prepare.cc:1378][EVENT]167201 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.375.744 [graph_manager.cc:1050][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [193] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.375.766 [graph_manager.cc:1052][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.375.918 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.375.948 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.375.997 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.376.009 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.376.054 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.376.068 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.376.085 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.376.174 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.376.211 [graph_manager.cc:1054][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [433] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.376.451 [graph_manager.cc:1055][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [226] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.377.523 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:34.377.551 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.377.563 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.377.573 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [343] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.377.582 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.377.590 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [4] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:34.377.600 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.377.608 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.377.616 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.379.472 [graph_manager.cc:1056][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3003] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.379.539 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.379.558 [graph_prepare.cc:1982][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [52] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.379.989 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:34.380.011 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.023 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.032 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [242] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.042 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.050 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:34.380.059 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.067 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.084 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.111 [graph_prepare.cc:1983][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [539] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.380.134 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.380.144 [graph_prepare.cc:1984][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.380.158 [graph_prepare.cc:1985][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.380.173 [graph_prepare.cc:1986][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.380.184 [graph_prepare.cc:1987][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.380.198 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.380.208 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.380.222 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.380.313 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.324 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.333 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.341 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.350 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.358 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.367 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.375 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.383 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.392 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.400 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.408 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.417 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.431 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [8] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.439 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.448 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.380.470 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.380.485 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.380.521 [graph_prepare.cc:1988][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [328] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.380.537 [graph_manager.cc:1065][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1031] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.393.378 [graph_manager.cc:1077][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12819] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.393.442 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.393.488 [graph_manager.cc:1080][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [74] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.396.653 [graph_manager.cc:1081][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3148] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.396.695 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.396.711 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.396.722 [graph_manager.cc:1082][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [34] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.396.754 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.396.769 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.396.783 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.396.886 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [93] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.396.903 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.396.953 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [41] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.396.968 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.022 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [30] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.045 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.071 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.099 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.115 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.127 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.136 [graph_manager.cc:2700][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [387] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.257 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.397.272 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.397.282 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.397.291 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.397.299 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.397.308 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.397.316 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.397.324 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.397.333 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.397.341 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.397.349 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.397.358 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.397.366 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.397.374 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.397.383 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.397.392 [graph_manager.cc:2741][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [239] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.408 [graph_manager.cc:2752][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.431 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.443 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.459 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.473 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.485 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.498 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.516 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.529 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.542 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.552 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.565 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.576 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.595 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.607 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.616 [graph_manager.cc:2810][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [189] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.644 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.397.655 [graph_manager.cc:2821][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.684 [graph_manager.cc:1087][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [943] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.842 [graph_manager.cc:1088][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [118] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.884 [graph_manager.cc:1089][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.903 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.397.926 [graph_manager.cc:1097][EVENT]167201 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:34.397.948 [graph_manager.cc:3325][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.398.317 [engine_place.cc:144][EVENT]167201 Run:The time cost of AIcoreEngine::CheckSupported is [268] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.398.341 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.398.351 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.398.438 [graph_manager.cc:3351][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [475] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.398.457 [graph_manager.cc:3364][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.398.524 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.398.542 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.398.697 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [146] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.398.739 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [29] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.398.785 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [34] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.398.819 [graph_manager.cc:3405][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [351] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.398.838 [graph_manager.cc:3412][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.496.957 [graph_manager.cc:3422][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [98102] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.024 [graph_manager.cc:3428][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.209 [graph_manager.cc:3467][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [163] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.229 [graph_manager.cc:3377][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [98761] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.247 [graph_manager.cc:1106][EVENT]167201 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [99305] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.259 [graph_manager.cc:1115][EVENT]167201 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:34.497.285 [graph_manager.cc:1130][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.334 [graph_manager.cc:1131][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.363 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.380 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.390 [graph_manager.cc:2837][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.497 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.497.509 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.497.518 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.497.527 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.497.536 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.497.544 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [9] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.497.555 [graph_manager.cc:2864][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [146] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.567 [graph_manager.cc:2872][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.586 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.600 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.615 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.628 [compile_nodes_pass.cc:88][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.638 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.648 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.746 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [89] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.792 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [32] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.806 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.826 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.841 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.850 [graph_manager.cc:2927][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [269] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.862 [graph_manager.cc:2937][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.877 [graph_manager.cc:2943][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.497.888 [graph_manager.cc:2950][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.498.099 [graph_manager.cc:2958][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.498.131 [graph_manager.cc:1132][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [783] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.498.211 [graph_manager.cc:1135][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [66] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.498.246 [graph_manager.cc:2975][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.498.278 [graph_manager.cc:2981][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.498.293 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.498.303 [graph_manager.cc:2986][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.498.312 [graph_manager.cc:1136][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [85] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.498.455 [graph_manager.cc:3555][EVENT]167201 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [109] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.498.556 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.498.574 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.498.713 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [128] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.498.750 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [24] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.498.793 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [32] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.498.827 [graph_builder.cc:865][EVENT]167201 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [311] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:34.499.194 [logger.cc:1071] 167201 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:37:34.499.226 [task_generator.cc:804][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [97] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.499.289 [task_generator.cc:805][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [49] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.500.234 [task_generator.cc:814][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [930] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.500.250 [task_generator.cc:954][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1122] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.500.320 [task_generator.cc:967][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [40] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:34.500.342 [logger.cc:1084] 167201 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:37:34.502.224 [graph_manager.cc:1152][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3884] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.502.272 [graph_manager.cc:1164][EVENT]167201 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:34.502.314 [graph_manager.cc:1271][EVENT]167201 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [126918] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.502.325 [graph_manager.cc:1272][EVENT]167201 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:37:34.502.653 [atrace_api.c:93](tid:167201) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:37:34.502.671 [atrace_api.c:95](tid:167201) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:37:34.533.814 [graph_converter.cc:838][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [10085] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.534.017 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [142] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.534.766 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [728] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.535.016 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [225] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.535.038 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [248] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.535.239 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [189] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.535.289 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [32] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.535.330 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [27] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.535.594 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [252] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.535.704 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [91] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.535.719 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [107] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.535.780 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [28] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.535.813 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.535.846 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.535.957 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [101] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.536.050 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [81] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.536.062 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [93] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.536.096 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [24] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.536.125 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.536.139 [graph_converter.cc:849][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [2270] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.536.478 [graph_converter.cc:853][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [330] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.537.450 [graph_converter.cc:857][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [955] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.537.648 [graph_converter.cc:862][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [172] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.621.311 [graph_var_manager.cc:1424][EVENT]167203 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:34.621.401 [graph_manager.cc:1248][EVENT]167203 PreRun:PreRun start: graph node size 4, session id 14, graph id 13, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:34.621.664 [atrace_api.c:28](tid:167203) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:34.621.731 [trace_rb_log.c:84](tid:167203) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:34.621.746 [atrace_api.c:32](tid:167203) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:34.621.765 [client_manager.cpp:157][SetProfilingCallback][tid:167203] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:34.622.187 [parallel_partitioner.cc:165][EVENT]167203 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.622.224 [parallel_partitioner.cc:178][EVENT]167203 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.622.269 [graph_prepare.cc:1378][EVENT]167203 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.622.449 [graph_manager.cc:1050][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [197] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.622.478 [graph_manager.cc:1052][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.622.639 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.622.669 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.622.718 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.622.731 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.622.776 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.622.789 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.622.808 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.622.896 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.622.916 [graph_manager.cc:1054][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [406] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.623.154 [graph_manager.cc:1055][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [225] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.624.266 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:34.624.293 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.624.305 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.624.315 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferShapePass is [439] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.624.324 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.624.333 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:34.624.341 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.624.350 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.624.358 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.626.932 [graph_manager.cc:1056][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3757] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.627.001 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.627.020 [graph_prepare.cc:1982][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.627.588 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:34.627.613 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.627.623 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.627.633 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferShapePass is [350] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.627.642 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.627.650 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:34.627.659 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.627.667 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.627.676 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.627.702 [graph_prepare.cc:1983][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [656] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.627.725 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.627.737 [graph_prepare.cc:1984][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.627.751 [graph_prepare.cc:1985][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.627.765 [graph_prepare.cc:1986][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.627.775 [graph_prepare.cc:1987][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.627.790 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.627.800 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.627.814 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.627.905 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.627.916 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.627.925 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.627.934 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.627.942 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.627.958 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.627.967 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.627.976 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.627.984 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.627.993 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.628.001 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.628.009 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SnapshotPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.628.017 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.628.026 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.628.034 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.628.042 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.628.064 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.628.076 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.628.109 [graph_prepare.cc:1988][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [326] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.628.121 [graph_manager.cc:1065][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1156] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.641.001 [graph_manager.cc:1077][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12860] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.641.072 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.641.123 [graph_manager.cc:1080][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [85] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.651.599 [graph_manager.cc:1081][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [10457] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.651.656 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.651.673 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.651.685 [graph_manager.cc:1082][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.651.734 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.651.749 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.651.763 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.651.915 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [141] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.651.932 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.023 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [80] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.038 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.088 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.109 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.130 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.217 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [76] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.236 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.248 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.258 [graph_manager.cc:2700][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [529] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.494 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.652.509 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.652.519 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.652.528 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.652.537 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.652.546 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CastRemovePass is [40] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.652.554 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.652.563 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.652.571 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [10] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.652.587 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.652.596 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.652.604 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.652.613 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.652.621 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [8] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.652.629 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [6] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.652.640 [graph_manager.cc:2741][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [363] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.649 [graph_manager.cc:2752][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.673 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.685 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.706 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.722 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.734 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.747 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.767 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.781 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.794 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.805 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.817 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.830 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.851 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.864 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.879 [graph_manager.cc:2810][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [211] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.922 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.652.933 [graph_manager.cc:2821][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [46] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.652.962 [graph_manager.cc:1087][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1240] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.653.532 [graph_manager.cc:1088][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [557] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.653.592 [graph_manager.cc:1089][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [32] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.653.614 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.653.632 [graph_manager.cc:1097][EVENT]167203 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:34.653.655 [graph_manager.cc:3325][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.663.965 [engine_place.cc:144][EVENT]167203 Run:The time cost of AIcoreEngine::CheckSupported is [10071] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.663.997 [engine_place.cc:144][EVENT]167203 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.664.011 [engine_place.cc:144][EVENT]167203 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.664.107 [graph_manager.cc:3351][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10438] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.664.129 [graph_manager.cc:3364][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.664.206 [engine_partitioner.cc:1139][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [28] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.664.237 [engine_partitioner.cc:1142][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.664.412 [engine_partitioner.cc:1148][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [164] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.664.456 [engine_partitioner.cc:1155][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [27] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.664.504 [engine_partitioner.cc:1164][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.664.539 [graph_manager.cc:3405][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [398] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.664.558 [graph_manager.cc:3412][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.779.765 [graph_manager.cc:3422][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [115181] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.779.816 [graph_manager.cc:3428][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.779.984 [graph_manager.cc:3467][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [147] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.005 [graph_manager.cc:3377][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [115864] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.022 [graph_manager.cc:1106][EVENT]167203 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [126375] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.035 [graph_manager.cc:1115][EVENT]167203 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:34.780.061 [graph_manager.cc:1130][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.095 [graph_manager.cc:1131][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.124 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.144 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.154 [graph_manager.cc:2837][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [42] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.291 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [25] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.780.304 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.780.313 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.780.323 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.780.332 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [9] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.780.340 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [17] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.780.350 [graph_manager.cc:2864][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [179] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.363 [graph_manager.cc:2872][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.383 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.398 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.428 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.443 [compile_nodes_pass.cc:88][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.453 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.463 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.572 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [99] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.622 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.635 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.649 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.664 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.673 [graph_manager.cc:2927][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [293] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.686 [graph_manager.cc:2937][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.701 [graph_manager.cc:2943][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.712 [graph_manager.cc:2950][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.922 [graph_manager.cc:2958][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [54] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.780.956 [graph_manager.cc:1132][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [847] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.781.035 [graph_manager.cc:1135][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [66] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.781.072 [graph_manager.cc:2975][EVENT]167203 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.781.104 [graph_manager.cc:2981][EVENT]167203 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.781.118 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.781.127 [graph_manager.cc:2986][EVENT]167203 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.781.137 [graph_manager.cc:1136][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [85] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.781.480 [graph_manager.cc:3555][EVENT]167203 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [296] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.781.611 [engine_partitioner.cc:1139][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [32] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.781.640 [engine_partitioner.cc:1142][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.781.806 [engine_partitioner.cc:1148][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [155] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.781.845 [engine_partitioner.cc:1155][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [24] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.781.888 [engine_partitioner.cc:1164][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.781.914 [graph_builder.cc:865][EVENT]167203 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [365] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:34.782.300 [logger.cc:1071] 167203 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:37:34.782.332 [task_generator.cc:804][EVENT]167203 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [88] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.782.415 [task_generator.cc:805][EVENT]167203 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [70] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.784.001 [task_generator.cc:814][EVENT]167203 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1571] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.784.016 [task_generator.cc:954][EVENT]167203 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1772] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.784.090 [task_generator.cc:967][EVENT]167203 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [40] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:34.784.115 [logger.cc:1084] 167203 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:37:34.785.437 [graph_manager.cc:1152][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4258] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.785.474 [graph_manager.cc:1164][EVENT]167203 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:34.785.514 [graph_manager.cc:1271][EVENT]167203 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [163419] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.785.525 [graph_manager.cc:1272][EVENT]167203 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:37:34.785.885 [atrace_api.c:93](tid:167203) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:37:34.785.904 [atrace_api.c:95](tid:167203) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:37:34.815.641 [graph_converter.cc:838][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [11277] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.815.871 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of ZeroCopy is [167] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.817.399 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CEM is [1502] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.817.835 [copy_flow_launch_fuse.cc:395][EVENT]167203 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [391] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.817.860 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [419] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.818.127 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [253] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.818.209 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [60] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.818.276 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of ZeroCopy is [50] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.818.722 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CEM is [431] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.818.936 [copy_flow_launch_fuse.cc:395][EVENT]167203 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [190] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.818.957 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [212] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.819.022 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [54] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.819.080 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [45] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.819.137 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of ZeroCopy is [46] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.819.352 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CEM is [203] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.819.539 [copy_flow_launch_fuse.cc:395][EVENT]167203 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [172] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.819.556 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [189] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.819.615 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [50] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.819.670 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [44] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.819.687 [graph_converter.cc:849][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [3989] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.820.382 [graph_converter.cc:853][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [685] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.822.315 [graph_converter.cc:857][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1909] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.822.692 [graph_converter.cc:862][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [349] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.898.993 [graph_var_manager.cc:1424][EVENT]167201 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:34.899.081 [graph_manager.cc:1248][EVENT]167201 PreRun:PreRun start: graph node size 4, session id 15, graph id 14, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:34.899.345 [atrace_api.c:28](tid:167201) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:34.899.397 [trace_rb_log.c:84](tid:167201) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:34.899.410 [atrace_api.c:32](tid:167201) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:34.899.429 [client_manager.cpp:157][SetProfilingCallback][tid:167201] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:34.899.835 [parallel_partitioner.cc:165][EVENT]167201 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.899.871 [parallel_partitioner.cc:178][EVENT]167201 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.899.919 [graph_prepare.cc:1378][EVENT]167201 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.900.093 [graph_manager.cc:1050][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [192] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.900.117 [graph_manager.cc:1052][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.900.258 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.900.289 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.900.338 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.900.351 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.900.398 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.900.410 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.900.428 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.900.514 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.900.535 [graph_manager.cc:1054][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [404] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.900.775 [graph_manager.cc:1055][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [227] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.901.918 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:34.901.945 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.901.956 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.901.965 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [467] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.901.974 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.901.992 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:34.902.002 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.902.010 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.902.019 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.904.474 [graph_manager.cc:1056][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3680] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.904.543 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.904.561 [graph_prepare.cc:1982][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.905.116 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:34.905.141 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.155 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.167 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [349] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.176 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.185 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:34.905.193 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.201 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.210 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.236 [graph_prepare.cc:1983][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [660] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.905.260 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.905.271 [graph_prepare.cc:1984][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.905.285 [graph_prepare.cc:1985][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.905.300 [graph_prepare.cc:1986][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.905.310 [graph_prepare.cc:1987][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.905.334 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.905.347 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.905.361 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.905.452 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.463 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.472 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.481 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.490 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.501 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.511 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.522 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.531 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.539 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.547 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.555 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SnapshotPass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.564 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.572 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.580 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.589 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:34.905.612 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.905.625 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.905.658 [graph_prepare.cc:1988][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [329] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.905.671 [graph_manager.cc:1065][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1163] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.918.626 [graph_manager.cc:1077][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12900] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.918.698 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.918.747 [graph_manager.cc:1080][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [84] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.929.217 [graph_manager.cc:1081][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [10453] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.929.274 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.929.290 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.929.302 [graph_manager.cc:1082][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.929.334 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.929.349 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.929.363 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.929.511 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [139] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.929.528 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.929.618 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [78] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.929.633 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.929.683 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [40] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.929.735 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.929.758 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.929.846 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [78] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.929.864 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.929.877 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.929.887 [graph_manager.cc:2700][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [559] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.930.123 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.930.155 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AddNPass is [3] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.930.165 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.930.174 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.930.183 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.930.191 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CastRemovePass is [38] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.930.200 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.930.208 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.930.216 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [9] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.930.225 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [5] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.930.233 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [17] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.930.242 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.930.250 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.930.259 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [12] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.930.267 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.930.277 [graph_manager.cc:2741][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [371] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.930.286 [graph_manager.cc:2752][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.930.312 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.930.324 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.930.345 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.930.362 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.930.374 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.930.386 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.930.406 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.930.426 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.930.439 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.930.449 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.930.461 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.930.473 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.930.493 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.930.506 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.930.515 [graph_manager.cc:2810][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [208] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.930.558 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:34.930.569 [graph_manager.cc:2821][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [46] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.930.598 [graph_manager.cc:1087][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1277] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.931.172 [graph_manager.cc:1088][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [561] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.931.233 [graph_manager.cc:1089][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.931.254 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.931.271 [graph_manager.cc:1097][EVENT]167201 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:34.931.294 [graph_manager.cc:3325][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.941.441 [engine_place.cc:144][EVENT]167201 Run:The time cost of AIcoreEngine::CheckSupported is [9927] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.941.473 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.941.484 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.941.577 [graph_manager.cc:3351][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10268] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.941.597 [graph_manager.cc:3364][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.941.676 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [28] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.941.740 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.941.917 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [163] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.941.960 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [29] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.942.010 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.942.048 [graph_manager.cc:3405][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [438] micro second. [INFO] GE(164039,python):2024-01-10-11:37:34.942.067 [graph_manager.cc:3412][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.056.930 [graph_manager.cc:3422][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [114846] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.056.992 [graph_manager.cc:3428][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.181 [graph_manager.cc:3467][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [163] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.201 [graph_manager.cc:3377][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [115591] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.220 [graph_manager.cc:1106][EVENT]167201 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [125932] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.233 [graph_manager.cc:1115][EVENT]167201 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:35.057.259 [graph_manager.cc:1130][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.293 [graph_manager.cc:1131][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.323 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.344 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.353 [graph_manager.cc:2837][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [43] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.494 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [28] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:35.057.507 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:35.057.516 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:35.057.539 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of BitcastPass is [3] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:35.057.548 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:35.057.557 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [14] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:35.057.568 [graph_manager.cc:2864][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [196] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.580 [graph_manager.cc:2872][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.599 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.614 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.630 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.644 [compile_nodes_pass.cc:88][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.654 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.664 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.795 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [123] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.848 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.862 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.876 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.891 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.901 [graph_manager.cc:2927][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [305] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.913 [graph_manager.cc:2937][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.929 [graph_manager.cc:2943][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.057.940 [graph_manager.cc:2950][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.058.152 [graph_manager.cc:2958][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [56] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.058.196 [graph_manager.cc:1132][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [888] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.058.281 [graph_manager.cc:1135][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [72] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.058.320 [graph_manager.cc:2975][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.058.353 [graph_manager.cc:2981][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.058.368 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.058.377 [graph_manager.cc:2986][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.058.386 [graph_manager.cc:1136][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [89] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.058.724 [graph_manager.cc:3555][EVENT]167201 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [297] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.058.856 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [33] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.058.885 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.059.030 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [134] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.059.066 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.059.108 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.059.134 [graph_builder.cc:865][EVENT]167201 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [342] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:35.059.530 [logger.cc:1071] 167201 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:37:35.059.562 [task_generator.cc:804][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [88] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.059.645 [task_generator.cc:805][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [71] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.061.265 [task_generator.cc:814][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1605] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.061.279 [task_generator.cc:954][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1807] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.061.352 [task_generator.cc:967][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [39] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:35.061.378 [logger.cc:1084] 167201 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:37:35.062.706 [graph_manager.cc:1152][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4284] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.062.747 [graph_manager.cc:1164][EVENT]167201 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:35.062.787 [graph_manager.cc:1271][EVENT]167201 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [163043] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.062.798 [graph_manager.cc:1272][EVENT]167201 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:37:35.063.121 [atrace_api.c:93](tid:167201) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:37:35.063.139 [atrace_api.c:95](tid:167201) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:37:35.091.838 [graph_converter.cc:838][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [10647] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.092.064 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [166] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.093.563 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [1473] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.093.980 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [388] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.094.006 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [416] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.094.274 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [255] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.094.359 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [63] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.094.426 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [50] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.094.884 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [444] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.095.100 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [191] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.095.121 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [212] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.095.187 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [55] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.095.244 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [45] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.095.302 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [47] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.095.522 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [209] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.095.717 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [179] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.095.732 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [196] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.095.794 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [52] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.095.850 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [46] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.095.868 [graph_converter.cc:849][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [3977] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.096.591 [graph_converter.cc:853][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [696] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.098.537 [graph_converter.cc:857][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1924] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.098.918 [graph_converter.cc:862][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [352] micro second. [INFO] HCCP(164039,python):2024-01-10-11:37:35.116.211 [ra_host.c:1761]tid:168539,ra_socket_white_list_add(1761) : Input parameters: phy_id[0], local_ip[0.0.0.0], num[1] [INFO] HCCP(164039,python):2024-01-10-11:37:35.116.464 [ra_host.c:1761]tid:168539,ra_socket_white_list_add(1761) : Input parameters: phy_id[0], local_ip[0.0.0.0], num[1] [INFO] HCCP(164039,python):2024-01-10-11:37:35.116.558 [ra_host.c:825]tid:168539,ra_socket_batch_connect(825) : Input parameters: [0]th, phy_id[0], local_ip[0.0.0.0], remote_ip[2.0.0.0], tag:[8.92.9.85%enp189s0f0_60000_0_1704857845691656] [INFO] HCCP(164039,python):2024-01-10-11:37:35.508.951 [ra_host.c:1761]tid:167202,ra_socket_white_list_add(1761) : Input parameters: phy_id[0], local_ip[0.0.0.0], num[1] [INFO] HCCP(164039,python):2024-01-10-11:37:35.509.097 [ra_host.c:825]tid:167202,ra_socket_batch_connect(825) : Input parameters: [0]th, phy_id[0], local_ip[0.0.0.0], remote_ip[2.0.0.0], tag:[HeartBeat_8.92.9.85/0_to_8.92.9.85/2] [INFO] HCCL(164039,python):2024-01-10-11:37:35.516.135 [hccl_impl.cc:3258][167202]resource creation success, take time [401337]us, tag[AllReduce_8.92.9.85%enp189s0f0_60000_0_1704857845691656] [INFO] GE(164039,python):2024-01-10-11:37:35.580.160 [graph_var_manager.cc:1424][EVENT]167202 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:35.580.246 [graph_manager.cc:1248][EVENT]167202 PreRun:PreRun start: graph node size 3, session id 16, graph id 15, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:35.580.493 [atrace_api.c:28](tid:167202) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:35.580.525 [trace_rb_log.c:84](tid:167202) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:35.580.538 [atrace_api.c:32](tid:167202) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:35.580.558 [client_manager.cpp:157][SetProfilingCallback][tid:167202] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:35.580.946 [parallel_partitioner.cc:165][EVENT]167202 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.580.981 [parallel_partitioner.cc:178][EVENT]167202 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.581.028 [graph_prepare.cc:1378][EVENT]167202 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.581.209 [graph_manager.cc:1050][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [198] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.581.232 [graph_manager.cc:1052][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.581.354 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.581.400 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.581.448 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.581.461 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.581.505 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.581.517 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.581.534 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.581.619 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.581.638 [graph_manager.cc:1054][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [393] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.581.898 [graph_manager.cc:1055][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [247] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.582.731 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:35.582.757 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.582.768 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.582.778 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferShapePass is [265] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.582.787 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.582.795 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:35.582.804 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [10] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.582.812 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.582.820 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.584.452 [graph_manager.cc:1056][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2532] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.584.517 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.584.536 [graph_prepare.cc:1982][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [50] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.584.911 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:35.584.935 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.584.967 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.584.977 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferShapePass is [173] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.584.986 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.584.994 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [6] [INFO] GE(164039,python):2024-01-10-11:37:35.585.003 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.585.011 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [8] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.585.019 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferValuePass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.585.071 [graph_prepare.cc:1983][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [521] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.585.096 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.585.107 [graph_prepare.cc:1984][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.585.121 [graph_prepare.cc:1985][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.585.135 [graph_prepare.cc:1986][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.585.146 [graph_prepare.cc:1987][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.585.161 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.585.172 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.585.184 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.585.264 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.585.275 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.585.284 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.585.292 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.585.301 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.585.309 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.585.324 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.585.333 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.585.342 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of StopGradientPass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.585.350 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.585.358 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.585.366 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.585.375 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.585.383 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.585.391 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.585.399 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.585.421 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.585.432 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.585.462 [graph_prepare.cc:1988][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [306] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.585.475 [graph_manager.cc:1065][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [989] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.597.337 [graph_manager.cc:1077][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11839] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.597.437 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.597.485 [graph_manager.cc:1080][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [111] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.600.535 [graph_manager.cc:1081][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3033] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.600.579 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.600.595 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.600.609 [graph_manager.cc:1082][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.600.639 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.600.669 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.600.683 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.600.714 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.600.730 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.600.743 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.600.756 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.600.793 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [28] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.600.810 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.600.828 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.600.854 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.600.872 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.600.883 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.600.892 [graph_manager.cc:2700][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [258] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.600.997 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.601.011 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.601.021 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.601.029 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [0] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.601.038 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.601.047 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CastRemovePass is [9] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.601.056 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.601.064 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.601.073 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.601.081 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.601.096 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.601.105 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.601.114 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.601.122 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.601.130 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.601.140 [graph_manager.cc:2741][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [230] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.149 [graph_manager.cc:2752][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.171 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.182 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.198 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.212 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.224 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.236 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.253 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.266 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.278 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.288 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.302 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.314 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.332 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.344 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.353 [graph_manager.cc:2810][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [186] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.396 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.601.407 [graph_manager.cc:2821][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [28] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.435 [graph_manager.cc:1087][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [808] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.564 [graph_manager.cc:1088][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [117] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.601 [graph_manager.cc:1089][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.618 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.601.633 [graph_manager.cc:1097][EVENT]167202 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:35.601.654 [graph_manager.cc:3325][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.602.010 [engine_place.cc:144][EVENT]167202 Run:The time cost of AIcoreEngine::CheckSupported is [248] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.602.038 [engine_place.cc:144][EVENT]167202 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.602.047 [engine_place.cc:144][EVENT]167202 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.602.120 [graph_manager.cc:3351][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [452] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.602.138 [graph_manager.cc:3364][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.602.198 [engine_partitioner.cc:1139][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.602.214 [engine_partitioner.cc:1142][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.602.343 [engine_partitioner.cc:1148][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [119] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.602.382 [engine_partitioner.cc:1155][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [27] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.602.426 [engine_partitioner.cc:1164][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [33] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.602.460 [graph_manager.cc:3405][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [310] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.602.478 [graph_manager.cc:3412][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.692.667 [graph_manager.cc:3422][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [90174] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.692.739 [graph_manager.cc:3428][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.692.899 [graph_manager.cc:3467][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [138] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.692.919 [graph_manager.cc:3377][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [90769] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.692.935 [graph_manager.cc:1106][EVENT]167202 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [91287] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.692.948 [graph_manager.cc:1115][EVENT]167202 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:35.692.974 [graph_manager.cc:1130][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.009 [graph_manager.cc:1131][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.037 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.056 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.065 [graph_manager.cc:2837][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [40] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.159 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.693.171 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.693.181 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.693.189 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.693.198 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.693.206 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [6] micro second, call num is [3] [INFO] GE(164039,python):2024-01-10-11:37:35.693.216 [graph_manager.cc:2864][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [132] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.228 [graph_manager.cc:2872][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.248 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.261 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.276 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.296 [compile_nodes_pass.cc:88][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.307 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.316 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.392 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [66] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.434 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [30] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.447 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.460 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.474 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.483 [graph_manager.cc:2927][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [239] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.495 [graph_manager.cc:2937][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.509 [graph_manager.cc:2943][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.520 [graph_manager.cc:2950][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.735 [graph_manager.cc:2958][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [52] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.768 [graph_manager.cc:1132][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [746] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.847 [graph_manager.cc:1135][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [64] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.879 [graph_manager.cc:2975][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.910 [graph_manager.cc:2981][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.925 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.934 [graph_manager.cc:2986][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.693.943 [graph_manager.cc:1136][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [82] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.694.070 [graph_manager.cc:3555][EVENT]167202 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [95] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.694.168 [engine_partitioner.cc:1139][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.694.185 [engine_partitioner.cc:1142][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.694.287 [engine_partitioner.cc:1148][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [92] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.694.319 [engine_partitioner.cc:1155][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.694.359 [engine_partitioner.cc:1164][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [30] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.694.383 [graph_builder.cc:865][EVENT]167202 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [249] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:35.694.679 [logger.cc:1071] 167202 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:37:35.694.708 [task_generator.cc:804][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [81] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.694.766 [task_generator.cc:805][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [45] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.695.521 [task_generator.cc:814][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [743] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.695.535 [task_generator.cc:954][EVENT]167202 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [908] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.695.601 [task_generator.cc:967][EVENT]167202 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [37] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:35.695.620 [logger.cc:1084] 167202 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:37:35.696.907 [graph_manager.cc:1152][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2939] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.696.943 [graph_manager.cc:1164][EVENT]167202 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:35.696.984 [graph_manager.cc:1271][EVENT]167202 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [116124] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.696.995 [graph_manager.cc:1272][EVENT]167202 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:37:35.697.322 [atrace_api.c:93](tid:167202) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:37:35.697.340 [atrace_api.c:95](tid:167202) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:37:35.721.038 [graph_converter.cc:838][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [7865] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.721.231 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [133] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.721.873 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [621] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.722.088 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [189] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.722.125 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [226] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.722.314 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [176] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.722.357 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [25] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.722.390 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.722.610 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [208] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.722.707 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [78] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.722.721 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [94] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.722.754 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.722.781 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.722.809 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.722.895 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [76] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.722.974 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [67] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.722.985 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [79] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.723.014 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.723.040 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.723.053 [graph_converter.cc:849][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1960] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.723.329 [graph_converter.cc:853][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [267] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.724.182 [graph_converter.cc:857][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [837] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.724.344 [graph_converter.cc:862][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [137] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.798.289 [graph_var_manager.cc:1424][EVENT]167203 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:35.798.376 [graph_manager.cc:1248][EVENT]167203 PreRun:PreRun start: graph node size 4, session id 17, graph id 16, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:35.798.665 [atrace_api.c:28](tid:167203) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:35.798.692 [trace_rb_log.c:84](tid:167203) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:35.798.705 [atrace_api.c:32](tid:167203) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:35.798.743 [client_manager.cpp:157][SetProfilingCallback][tid:167203] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:35.799.139 [parallel_partitioner.cc:165][EVENT]167203 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.799.175 [parallel_partitioner.cc:178][EVENT]167203 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.799.221 [graph_prepare.cc:1378][EVENT]167203 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.799.407 [graph_manager.cc:1050][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [202] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.799.432 [graph_manager.cc:1052][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.799.571 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.799.601 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.799.650 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.799.663 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.799.708 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.799.720 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.799.739 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.799.826 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.799.846 [graph_manager.cc:1054][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [401] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.800.089 [graph_manager.cc:1055][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [230] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.801.098 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:35.801.123 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.801.135 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of MergePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.801.144 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferShapePass is [328] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.801.153 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.801.162 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:35.801.179 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.801.188 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.801.196 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferValuePass is [8] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.802.984 [graph_manager.cc:1056][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2875] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.803.051 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.069 [graph_prepare.cc:1982][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.803.556 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:35.803.580 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.591 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.601 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferShapePass is [278] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.610 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.619 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:35.803.627 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.636 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.644 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.670 [graph_prepare.cc:1983][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [587] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.803.692 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.803.703 [graph_prepare.cc:1984][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.803.717 [graph_prepare.cc:1985][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.803.732 [graph_prepare.cc:1986][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.803.743 [graph_prepare.cc:1987][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.803.757 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.803.769 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.803.791 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.803.882 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.893 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.902 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.911 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.919 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.928 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.936 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.944 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.952 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.960 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.969 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.977 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.985 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.803.994 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.804.002 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.804.010 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.804.032 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.804.045 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.804.077 [graph_prepare.cc:1988][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [325] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.804.090 [graph_manager.cc:1065][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1074] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.816.348 [graph_manager.cc:1077][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12239] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.816.420 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.816.479 [graph_manager.cc:1080][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [93] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.824.564 [graph_manager.cc:1081][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [8070] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.824.619 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.824.635 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.824.647 [graph_manager.cc:1082][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.824.679 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.824.692 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.824.706 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.824.801 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [85] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.824.817 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.824.861 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [34] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.824.877 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.824.918 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [29] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.824.940 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.824.970 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.824.998 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.017 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.029 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.039 [graph_manager.cc:2700][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [365] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.169 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.825.183 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AddNPass is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.825.212 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.825.221 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.825.230 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.825.239 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.825.247 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.825.256 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.825.264 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.825.272 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.825.281 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.825.289 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.825.297 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.825.305 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.825.313 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.825.325 [graph_manager.cc:2741][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [265] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.337 [graph_manager.cc:2752][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.360 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.371 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.389 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.406 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.420 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.435 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.454 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.468 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.486 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.497 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.509 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.520 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.539 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.552 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.561 [graph_manager.cc:2810][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [205] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.590 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.825.601 [graph_manager.cc:2821][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.629 [graph_manager.cc:1087][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [963] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.788 [graph_manager.cc:1088][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [146] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.831 [graph_manager.cc:1089][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.848 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.825.863 [graph_manager.cc:1097][EVENT]167203 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:35.825.885 [graph_manager.cc:3325][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.826.265 [engine_place.cc:144][EVENT]167203 Run:The time cost of AIcoreEngine::CheckSupported is [284] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.826.291 [engine_place.cc:144][EVENT]167203 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.826.301 [engine_place.cc:144][EVENT]167203 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.826.384 [graph_manager.cc:3351][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [487] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.826.405 [graph_manager.cc:3364][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.826.477 [engine_partitioner.cc:1139][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.826.498 [engine_partitioner.cc:1142][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.826.668 [engine_partitioner.cc:1148][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [153] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.826.712 [engine_partitioner.cc:1155][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [30] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.826.761 [engine_partitioner.cc:1164][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.826.794 [graph_manager.cc:3405][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [377] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.826.812 [graph_manager.cc:3412][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.039 [graph_manager.cc:3422][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [16213] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.084 [graph_manager.cc:3428][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.231 [graph_manager.cc:3467][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [125] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.249 [graph_manager.cc:3377][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [16832] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.266 [graph_manager.cc:1106][EVENT]167203 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [17388] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.278 [graph_manager.cc:1115][EVENT]167203 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:35.843.304 [graph_manager.cc:1130][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.341 [graph_manager.cc:1131][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.368 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.385 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.395 [graph_manager.cc:2837][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.484 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.843.496 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.843.505 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CondRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.843.514 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of BitcastPass is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.843.533 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.843.542 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [8] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.843.552 [graph_manager.cc:2864][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [141] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.564 [graph_manager.cc:2872][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.582 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.597 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.611 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.625 [compile_nodes_pass.cc:88][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.635 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.645 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.726 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [71] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.755 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.768 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.782 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.794 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.803 [graph_manager.cc:2927][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [223] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.816 [graph_manager.cc:2937][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.829 [graph_manager.cc:2943][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.843.840 [graph_manager.cc:2950][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.844.025 [graph_manager.cc:2958][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.844.055 [graph_manager.cc:1132][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [700] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.844.132 [graph_manager.cc:1135][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [56] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.844.165 [graph_manager.cc:2975][EVENT]167203 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.844.197 [graph_manager.cc:2981][EVENT]167203 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.844.211 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.844.220 [graph_manager.cc:2986][EVENT]167203 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.844.229 [graph_manager.cc:1136][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [81] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.844.364 [graph_manager.cc:3555][EVENT]167203 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [102] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.844.459 [engine_partitioner.cc:1139][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.844.476 [engine_partitioner.cc:1142][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.844.603 [engine_partitioner.cc:1148][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [114] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.844.641 [engine_partitioner.cc:1155][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.844.681 [engine_partitioner.cc:1164][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [30] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.844.705 [graph_builder.cc:865][EVENT]167203 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [283] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:35.845.012 [logger.cc:1071] 167203 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:37:35.845.043 [task_generator.cc:804][EVENT]167203 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [76] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.845.108 [task_generator.cc:805][EVENT]167203 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [50] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.845.804 [task_generator.cc:814][EVENT]167203 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [682] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.845.818 [task_generator.cc:954][EVENT]167203 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [852] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.845.878 [task_generator.cc:967][EVENT]167203 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [34] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:35.845.897 [logger.cc:1084] 167203 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:37:35.846.469 [graph_manager.cc:1152][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2212] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.846.499 [graph_manager.cc:1164][EVENT]167203 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:35.846.544 [graph_manager.cc:1271][EVENT]167203 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [47498] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.846.556 [graph_manager.cc:1272][EVENT]167203 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:37:35.846.871 [atrace_api.c:93](tid:167203) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:37:35.846.888 [atrace_api.c:95](tid:167203) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:37:35.857.981 [graph_converter.cc:838][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [3631] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.858.152 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of ZeroCopy is [122] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.858.635 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CEM is [459] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.858.848 [copy_flow_launch_fuse.cc:395][EVENT]167203 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [188] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.858.869 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [212] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.859.092 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [210] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.859.132 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.859.162 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.859.355 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CEM is [182] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.859.439 [copy_flow_launch_fuse.cc:395][EVENT]167203 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [66] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.859.453 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [81] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.859.482 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.859.507 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.859.533 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.859.607 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CEM is [65] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.859.676 [copy_flow_launch_fuse.cc:395][EVENT]167203 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [57] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.859.687 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [70] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.859.713 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.859.736 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.859.749 [graph_converter.cc:849][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1726] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.859.972 [graph_converter.cc:853][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [213] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.860.693 [graph_converter.cc:857][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [696] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.860.839 [graph_converter.cc:862][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [122] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.925.857 [graph_var_manager.cc:1424][EVENT]167201 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:35.925.946 [graph_manager.cc:1248][EVENT]167201 PreRun:PreRun start: graph node size 4, session id 18, graph id 17, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:35.926.208 [atrace_api.c:28](tid:167201) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:35.926.235 [trace_rb_log.c:84](tid:167201) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:35.926.248 [atrace_api.c:32](tid:167201) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:35.926.267 [client_manager.cpp:157][SetProfilingCallback][tid:167201] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:35.926.679 [parallel_partitioner.cc:165][EVENT]167201 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.926.717 [parallel_partitioner.cc:178][EVENT]167201 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.926.765 [graph_prepare.cc:1378][EVENT]167201 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.926.950 [graph_manager.cc:1050][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [203] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.926.976 [graph_manager.cc:1052][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.927.121 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.927.150 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.927.198 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.927.211 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.927.256 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.927.270 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.927.288 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.927.374 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.927.394 [graph_manager.cc:1054][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [403] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.927.662 [graph_manager.cc:1055][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [237] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.928.667 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [5] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:35.928.693 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.928.704 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.928.714 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [335] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.928.724 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.928.732 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [5] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:35.928.741 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.928.750 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.928.758 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.930.564 [graph_manager.cc:1056][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2882] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.930.632 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.930.650 [graph_prepare.cc:1982][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [54] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.931.102 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:35.931.126 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.137 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.146 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [256] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.155 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.163 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:35.931.172 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.180 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.188 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.215 [graph_prepare.cc:1983][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [551] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.931.248 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.931.260 [graph_prepare.cc:1984][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.931.275 [graph_prepare.cc:1985][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.931.289 [graph_prepare.cc:1986][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.931.299 [graph_prepare.cc:1987][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.931.312 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.931.324 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.931.338 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.931.428 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.439 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.448 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.457 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.465 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.473 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.482 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.490 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.499 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.507 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.515 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.523 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.532 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.540 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.548 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.561 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.931.585 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.931.598 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.931.632 [graph_prepare.cc:1988][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [324] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.931.644 [graph_manager.cc:1065][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1049] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.943.648 [graph_manager.cc:1077][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11984] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.943.724 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.943.777 [graph_manager.cc:1080][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [91] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.319 [graph_manager.cc:1081][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [4523] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.364 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.380 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.391 [graph_manager.cc:1082][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.424 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.440 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.454 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.484 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.497 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.511 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.523 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.562 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [30] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.580 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.614 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.642 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.657 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.669 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.677 [graph_manager.cc:2700][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [259] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.798 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.948.811 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.948.821 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.948.829 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.948.838 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.948.847 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.948.855 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.948.863 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.948.872 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.948.880 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.948.888 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.948.896 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.948.904 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.948.912 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.948.920 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.948.930 [graph_manager.cc:2741][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [234] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.939 [graph_manager.cc:2752][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.961 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.978 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.948.996 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.949.011 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.949.024 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.949.036 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.949.055 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.949.069 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.949.082 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.949.092 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.949.105 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.949.117 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.949.136 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.949.149 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.949.158 [graph_manager.cc:2810][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [200] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.949.187 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.949.198 [graph_manager.cc:2821][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.949.225 [graph_manager.cc:1087][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [814] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.949.357 [graph_manager.cc:1088][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [119] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.949.398 [graph_manager.cc:1089][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.949.417 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.949.432 [graph_manager.cc:1097][EVENT]167201 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:35.949.453 [graph_manager.cc:3325][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.949.981 [engine_place.cc:144][EVENT]167201 Run:The time cost of AIcoreEngine::CheckSupported is [422] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.950.007 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.950.016 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.950.095 [graph_manager.cc:3351][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [622] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.950.115 [graph_manager.cc:3364][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.950.187 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.950.205 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.950.366 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [152] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.950.412 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [33] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.950.460 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.950.494 [graph_manager.cc:3405][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [366] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.950.512 [graph_manager.cc:3412][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.967.957 [graph_manager.cc:3422][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [17430] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.011 [graph_manager.cc:3428][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.164 [graph_manager.cc:3467][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [130] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.184 [graph_manager.cc:3377][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [18058] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.200 [graph_manager.cc:1106][EVENT]167201 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [18754] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.212 [graph_manager.cc:1115][EVENT]167201 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:35.968.236 [graph_manager.cc:1130][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.268 [graph_manager.cc:1131][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.306 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.324 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.334 [graph_manager.cc:2837][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.420 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.968.433 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.968.442 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.968.451 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.968.459 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.968.468 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:35.968.477 [graph_manager.cc:2864][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [127] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.489 [graph_manager.cc:2872][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.508 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.522 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.536 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.550 [compile_nodes_pass.cc:88][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.560 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.570 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.650 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [72] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.680 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.693 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.706 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.724 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.733 [graph_manager.cc:2927][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [227] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.745 [graph_manager.cc:2937][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.759 [graph_manager.cc:2943][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.769 [graph_manager.cc:2950][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.952 [graph_manager.cc:2958][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.968.983 [graph_manager.cc:1132][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [687] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.969.058 [graph_manager.cc:1135][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [61] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.969.090 [graph_manager.cc:2975][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.969.123 [graph_manager.cc:2981][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.969.137 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.969.147 [graph_manager.cc:2986][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.969.156 [graph_manager.cc:1136][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [83] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.969.271 [graph_manager.cc:3555][EVENT]167201 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [81] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.969.365 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.969.380 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.969.499 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [109] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.969.531 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.969.570 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [29] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.969.592 [graph_builder.cc:865][EVENT]167201 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [264] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:35.969.899 [logger.cc:1071] 167201 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:37:35.969.938 [task_generator.cc:804][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [81] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.970.003 [task_generator.cc:805][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [52] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.970.631 [task_generator.cc:814][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [613] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.970.646 [task_generator.cc:954][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [791] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.970.706 [task_generator.cc:967][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [33] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:35.970.723 [logger.cc:1084] 167201 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:37:35.971.412 [graph_manager.cc:1152][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2229] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.971.444 [graph_manager.cc:1164][EVENT]167201 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:35.971.483 [graph_manager.cc:1271][EVENT]167201 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [44896] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.971.497 [graph_manager.cc:1272][EVENT]167201 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:37:35.971.816 [atrace_api.c:93](tid:167201) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:37:35.971.836 [atrace_api.c:95](tid:167201) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:37:35.985.063 [graph_converter.cc:838][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [4566] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.985.239 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [125] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.985.767 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [504] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.985.985 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [191] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.986.007 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [215] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.986.226 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [208] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.986.267 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.986.299 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.986.491 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [181] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.986.577 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [66] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.986.591 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [81] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.986.620 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.986.645 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.986.691 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.986.767 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [65] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.986.835 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [56] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.986.846 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [67] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.986.872 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.986.895 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.986.908 [graph_converter.cc:849][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1801] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.987.132 [graph_converter.cc:853][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [214] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.987.875 [graph_converter.cc:857][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [728] micro second. [INFO] GE(164039,python):2024-01-10-11:37:35.988.022 [graph_converter.cc:862][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [123] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.056.077 [graph_var_manager.cc:1424][EVENT]167202 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:36.056.165 [graph_manager.cc:1248][EVENT]167202 PreRun:PreRun start: graph node size 4, session id 19, graph id 18, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:36.056.459 [atrace_api.c:28](tid:167202) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:36.056.488 [trace_rb_log.c:84](tid:167202) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:36.056.502 [atrace_api.c:32](tid:167202) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:36.056.521 [client_manager.cpp:157][SetProfilingCallback][tid:167202] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:36.056.936 [parallel_partitioner.cc:165][EVENT]167202 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.056.972 [parallel_partitioner.cc:178][EVENT]167202 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.057.019 [graph_prepare.cc:1378][EVENT]167202 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.057.202 [graph_manager.cc:1050][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [201] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.057.228 [graph_manager.cc:1052][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.057.367 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.057.397 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.057.462 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.057.475 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.057.520 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.057.534 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.057.552 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.057.641 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.057.660 [graph_manager.cc:1054][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [420] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.057.923 [graph_manager.cc:1055][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [249] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.058.916 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:36.058.942 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.058.954 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.058.964 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferShapePass is [321] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.058.973 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.058.982 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:36.058.990 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.058.999 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.059.007 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferValuePass is [9] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.060.715 [graph_manager.cc:1056][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2771] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.060.781 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.060.800 [graph_prepare.cc:1982][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [54] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.061.262 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:36.061.285 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.305 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.315 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferShapePass is [261] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.324 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.333 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:36.061.341 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [9] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.350 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.358 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.385 [graph_prepare.cc:1983][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [571] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.061.410 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.061.421 [graph_prepare.cc:1984][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.061.435 [graph_prepare.cc:1985][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.061.449 [graph_prepare.cc:1986][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.061.461 [graph_prepare.cc:1987][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.061.475 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.061.485 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.061.499 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.061.588 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.599 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.607 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.616 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.625 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.633 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.641 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.656 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.665 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.673 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.682 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.728 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.738 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.746 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.755 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.763 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.061.786 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.061.798 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.061.831 [graph_prepare.cc:1988][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [361] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.061.844 [graph_manager.cc:1065][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1099] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.074.024 [graph_manager.cc:1077][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12159] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.074.090 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.074.138 [graph_manager.cc:1080][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [79] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.081.853 [graph_manager.cc:1081][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [7699] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.081.905 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.081.922 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.081.933 [graph_manager.cc:1082][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.081.966 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.081.980 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.009 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.102 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [83] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.119 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.164 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [33] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.179 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.219 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [29] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.238 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.265 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.294 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.309 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.321 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.329 [graph_manager.cc:2700][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [369] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.456 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.082.469 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.082.478 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.082.487 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.082.496 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.082.504 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CastRemovePass is [11] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.082.513 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.082.521 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.082.529 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.082.537 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.082.553 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.082.562 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.082.570 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.082.579 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.082.587 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.082.597 [graph_manager.cc:2741][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [248] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.606 [graph_manager.cc:2752][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.628 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.639 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.657 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.672 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.683 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.694 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.712 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.726 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.739 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.748 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.761 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.772 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.791 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.804 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.812 [graph_manager.cc:2810][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [189] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.841 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.082.858 [graph_manager.cc:2821][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.082.887 [graph_manager.cc:1087][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [934] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.083.022 [graph_manager.cc:1088][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [122] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.083.063 [graph_manager.cc:1089][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.083.080 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.083.095 [graph_manager.cc:1097][EVENT]167202 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:36.083.116 [graph_manager.cc:3325][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.083.495 [engine_place.cc:144][EVENT]167202 Run:The time cost of AIcoreEngine::CheckSupported is [281] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.083.519 [engine_place.cc:144][EVENT]167202 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.083.529 [engine_place.cc:144][EVENT]167202 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.083.611 [graph_manager.cc:3351][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [482] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.083.629 [graph_manager.cc:3364][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.083.695 [engine_partitioner.cc:1139][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.083.713 [engine_partitioner.cc:1142][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.083.872 [engine_partitioner.cc:1148][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [148] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.083.915 [engine_partitioner.cc:1155][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.083.962 [engine_partitioner.cc:1164][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.083.996 [graph_manager.cc:3405][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [353] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.084.014 [graph_manager.cc:3412][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.099.297 [graph_manager.cc:3422][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [15267] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.099.357 [graph_manager.cc:3428][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.099.527 [graph_manager.cc:3467][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [127] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.099.550 [graph_manager.cc:3377][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [15909] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.099.566 [graph_manager.cc:1106][EVENT]167202 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [16456] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.099.579 [graph_manager.cc:1115][EVENT]167202 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:36.099.602 [graph_manager.cc:1130][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.099.634 [graph_manager.cc:1131][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.099.662 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.099.679 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.099.689 [graph_manager.cc:2837][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.099.782 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.099.801 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.099.811 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.099.820 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.099.829 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.099.837 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [8] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.099.847 [graph_manager.cc:2864][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [140] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.099.859 [graph_manager.cc:2872][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.099.877 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.099.891 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.099.905 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.099.919 [compile_nodes_pass.cc:88][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.099.935 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.099.946 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.031 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [75] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.061 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.075 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.089 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.102 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.113 [graph_manager.cc:2927][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [239] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.125 [graph_manager.cc:2937][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.140 [graph_manager.cc:2943][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.151 [graph_manager.cc:2950][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.341 [graph_manager.cc:2958][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [40] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.371 [graph_manager.cc:1132][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [722] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.441 [graph_manager.cc:1135][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [57] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.474 [graph_manager.cc:2975][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.507 [graph_manager.cc:2981][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.521 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.531 [graph_manager.cc:2986][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.540 [graph_manager.cc:1136][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [83] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.675 [graph_manager.cc:3555][EVENT]167202 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [103] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.769 [engine_partitioner.cc:1139][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.794 [engine_partitioner.cc:1142][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.919 [engine_partitioner.cc:1148][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [114] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.954 [engine_partitioner.cc:1155][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.100.993 [engine_partitioner.cc:1164][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [29] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.101.016 [graph_builder.cc:865][EVENT]167202 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [282] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:36.101.335 [logger.cc:1071] 167202 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:37:36.101.366 [task_generator.cc:804][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [83] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.101.431 [task_generator.cc:805][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [51] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.102.084 [task_generator.cc:814][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [634] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.102.102 [task_generator.cc:954][EVENT]167202 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [819] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.102.161 [task_generator.cc:967][EVENT]167202 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [33] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:36.102.182 [logger.cc:1084] 167202 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:37:36.102.771 [graph_manager.cc:1152][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2205] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.102.804 [graph_manager.cc:1164][EVENT]167202 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:36.102.842 [graph_manager.cc:1271][EVENT]167202 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [45999] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.102.853 [graph_manager.cc:1272][EVENT]167202 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:37:36.103.168 [atrace_api.c:93](tid:167202) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:37:36.103.186 [atrace_api.c:95](tid:167202) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:37:36.114.445 [graph_converter.cc:838][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [3741] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.114.614 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [121] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.115.086 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [452] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.115.302 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [190] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.115.322 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [212] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.115.557 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [211] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.115.596 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.115.626 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.115.821 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [182] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.115.906 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [67] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.115.919 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [80] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.115.949 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.115.973 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.115.999 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.116.073 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [66] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.116.139 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [55] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.116.149 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [66] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.116.175 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.116.197 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.116.210 [graph_converter.cc:849][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1723] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.116.435 [graph_converter.cc:853][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [215] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.117.147 [graph_converter.cc:857][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [697] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.117.291 [graph_converter.cc:862][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [120] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.185.345 [graph_var_manager.cc:1424][EVENT]167204 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:36.185.429 [graph_manager.cc:1248][EVENT]167204 PreRun:PreRun start: graph node size 4, session id 20, graph id 19, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:36.185.720 [atrace_api.c:28](tid:167204) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:36.185.749 [trace_rb_log.c:84](tid:167204) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:36.185.762 [atrace_api.c:32](tid:167204) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:36.185.782 [client_manager.cpp:157][SetProfilingCallback][tid:167204] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:36.186.200 [parallel_partitioner.cc:165][EVENT]167204 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.186.236 [parallel_partitioner.cc:178][EVENT]167204 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.186.283 [graph_prepare.cc:1378][EVENT]167204 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.186.431 [graph_manager.cc:1050][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [167] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.186.454 [graph_manager.cc:1052][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.186.593 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.186.623 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.186.670 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.186.684 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.186.729 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.186.742 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.186.759 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.186.846 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.186.867 [graph_manager.cc:1054][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [401] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.187.104 [graph_manager.cc:1055][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [223] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.188.214 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:36.188.241 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.188.252 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.188.262 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferShapePass is [438] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.188.271 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.188.280 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:36.188.288 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.188.305 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.188.314 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.190.762 [graph_manager.cc:1056][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3638] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.190.831 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.190.849 [graph_prepare.cc:1982][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [52] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.191.402 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:36.191.428 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.439 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.449 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferShapePass is [347] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.458 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.467 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:37:36.191.475 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.483 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.492 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.519 [graph_prepare.cc:1983][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [655] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.191.544 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.191.555 [graph_prepare.cc:1984][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.191.569 [graph_prepare.cc:1985][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.191.583 [graph_prepare.cc:1986][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.191.594 [graph_prepare.cc:1987][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.191.608 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.191.619 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.191.644 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.191.735 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.746 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.755 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.764 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.772 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.780 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.789 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.797 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.805 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.813 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.822 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.830 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.838 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.846 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.854 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.862 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:37:36.191.884 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.191.898 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.191.931 [graph_prepare.cc:1988][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [328] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.191.943 [graph_manager.cc:1065][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1146] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.204.932 [graph_manager.cc:1077][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12969] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.205.030 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.205.089 [graph_manager.cc:1080][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [120] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.215.526 [graph_manager.cc:1081][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [10419] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.215.580 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.215.595 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.215.607 [graph_manager.cc:1082][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.215.639 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.215.655 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.215.669 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.215.819 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [140] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.215.837 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.215.928 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [80] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.215.943 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.215.992 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.014 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.034 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.121 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [76] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.139 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.152 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.162 [graph_manager.cc:2700][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [529] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.399 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.216.413 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.216.423 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.216.447 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.216.457 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.216.466 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CastRemovePass is [41] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.216.474 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.216.483 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [5] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.216.491 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [9] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.216.499 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.216.507 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [18] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.216.516 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.216.524 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.216.532 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [8] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.216.540 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.216.550 [graph_manager.cc:2741][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [368] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.559 [graph_manager.cc:2752][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.583 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.595 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.616 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.631 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.642 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.655 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.676 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.690 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.703 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.718 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.730 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.743 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.765 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.778 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.787 [graph_manager.cc:2810][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [208] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.830 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.216.842 [graph_manager.cc:2821][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [46] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.216.870 [graph_manager.cc:1087][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1243] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.217.439 [graph_manager.cc:1088][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [556] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.217.498 [graph_manager.cc:1089][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.217.519 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.217.538 [graph_manager.cc:1097][EVENT]167204 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:36.217.560 [graph_manager.cc:3325][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.226.874 [engine_place.cc:144][EVENT]167204 Run:The time cost of AIcoreEngine::CheckSupported is [9095] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.226.910 [engine_place.cc:144][EVENT]167204 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.226.932 [engine_place.cc:144][EVENT]167204 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.227.029 [graph_manager.cc:3351][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [9455] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.227.048 [graph_manager.cc:3364][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.227.126 [engine_partitioner.cc:1139][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [29] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.227.158 [engine_partitioner.cc:1142][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.227.338 [engine_partitioner.cc:1148][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [159] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.227.382 [engine_partitioner.cc:1155][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [27] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.227.431 [engine_partitioner.cc:1164][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.227.468 [graph_manager.cc:3405][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [407] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.227.486 [graph_manager.cc:3412][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.339.382 [graph_manager.cc:3422][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [111881] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.339.432 [graph_manager.cc:3428][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.339.603 [graph_manager.cc:3467][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [149] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.339.622 [graph_manager.cc:3377][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [112562] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.339.639 [graph_manager.cc:1106][EVENT]167204 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [122085] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.339.652 [graph_manager.cc:1115][EVENT]167204 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:36.339.677 [graph_manager.cc:1130][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.339.709 [graph_manager.cc:1131][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.339.737 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.339.758 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.339.768 [graph_manager.cc:2837][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [43] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.339.905 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [27] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.339.918 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.339.927 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.339.936 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of BitcastPass is [0] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.339.945 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.339.967 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [15] micro second, call num is [9] [INFO] GE(164039,python):2024-01-10-11:37:36.339.978 [graph_manager.cc:2864][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [192] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.339.991 [graph_manager.cc:2872][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.012 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.028 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.044 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.058 [compile_nodes_pass.cc:88][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.068 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.078 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.187 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [100] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.238 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.253 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.269 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.283 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.293 [graph_manager.cc:2927][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [285] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.305 [graph_manager.cc:2937][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.319 [graph_manager.cc:2943][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.330 [graph_manager.cc:2950][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.543 [graph_manager.cc:2958][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [56] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.576 [graph_manager.cc:1132][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [854] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.655 [graph_manager.cc:1135][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [66] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.701 [graph_manager.cc:2975][EVENT]167204 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.734 [graph_manager.cc:2981][EVENT]167204 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.748 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.758 [graph_manager.cc:2986][EVENT]167204 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.340.767 [graph_manager.cc:1136][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [86] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.341.087 [graph_manager.cc:3555][EVENT]167204 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [281] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.341.215 [engine_partitioner.cc:1139][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.341.244 [engine_partitioner.cc:1142][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.341.389 [engine_partitioner.cc:1148][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [133] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.341.426 [engine_partitioner.cc:1155][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.341.469 [engine_partitioner.cc:1164][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.341.495 [graph_builder.cc:865][EVENT]167204 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [341] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:36.341.926 [logger.cc:1071] 167204 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:37:36.341.961 [task_generator.cc:804][EVENT]167204 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [88] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.342.047 [task_generator.cc:805][EVENT]167204 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [73] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.343.625 [task_generator.cc:814][EVENT]167204 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1563] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.343.639 [task_generator.cc:954][EVENT]167204 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1767] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.343.713 [task_generator.cc:967][EVENT]167204 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [40] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:37:36.343.738 [logger.cc:1084] 167204 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:37:36.345.059 [graph_manager.cc:1152][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4258] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.345.099 [graph_manager.cc:1164][EVENT]167204 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:37:36.345.152 [graph_manager.cc:1271][EVENT]167204 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [159041] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.345.165 [graph_manager.cc:1272][EVENT]167204 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:37:36.345.489 [atrace_api.c:93](tid:167204) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:37:36.345.506 [atrace_api.c:95](tid:167204) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:37:36.374.710 [graph_converter.cc:838][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [10794] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.374.934 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of ZeroCopy is [165] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.376.412 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CEM is [1454] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.376.805 [copy_flow_launch_fuse.cc:395][EVENT]167204 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [365] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.376.830 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [391] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.377.098 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [254] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.377.182 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [62] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.377.249 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of ZeroCopy is [51] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.377.710 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CEM is [447] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.377.924 [copy_flow_launch_fuse.cc:395][EVENT]167204 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [188] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.377.944 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [211] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.378.009 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [54] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.378.065 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [44] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.378.122 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of ZeroCopy is [46] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.378.342 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CEM is [209] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.378.531 [copy_flow_launch_fuse.cc:395][EVENT]167204 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [174] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.378.548 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [191] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.378.609 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [51] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.378.664 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [44] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.378.682 [graph_converter.cc:849][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [3919] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.379.391 [graph_converter.cc:853][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [700] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.381.330 [graph_converter.cc:857][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1900] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.381.739 [graph_converter.cc:862][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [381] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.456.758 [graph_var_manager.cc:1424][EVENT]167204 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:37:36.456.844 [graph_manager.cc:1248][EVENT]167204 PreRun:PreRun start: graph node size 5, session id 21, graph id 20, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:37:36.457.125 [atrace_api.c:28](tid:167204) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:37:36.457.152 [trace_rb_log.c:84](tid:167204) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:37:36.457.165 [atrace_api.c:32](tid:167204) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:37:36.457.184 [client_manager.cpp:157][SetProfilingCallback][tid:167204] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:37:36.457.615 [parallel_partitioner.cc:165][EVENT]167204 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.457.654 [parallel_partitioner.cc:178][EVENT]167204 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.457.716 [graph_prepare.cc:1378][EVENT]167204 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.457.908 [graph_manager.cc:1050][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [225] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.457.935 [graph_manager.cc:1052][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.458.093 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.458.124 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.458.174 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.458.187 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.458.233 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.458.246 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.458.263 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.458.352 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.458.372 [graph_manager.cc:1054][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [426] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.458.617 [graph_manager.cc:1055][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [231] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.459.826 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssertPass is [5] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:36.459.852 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.459.863 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.459.873 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferShapePass is [411] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.459.882 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [15] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.459.891 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [5] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:36.459.900 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.459.909 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.459.917 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferValuePass is [7] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.461.972 [graph_manager.cc:1056][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3317] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.462.040 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.462.056 [graph_prepare.cc:1982][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.462.668 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:36.462.691 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.462.702 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.462.711 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferShapePass is [343] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.462.720 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.462.728 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:37:36.462.737 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.462.745 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.462.753 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.462.807 [graph_prepare.cc:1983][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [737] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.462.841 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.462.853 [graph_prepare.cc:1984][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.462.867 [graph_prepare.cc:1985][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.462.882 [graph_prepare.cc:1986][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.462.893 [graph_prepare.cc:1987][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.462.907 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.462.918 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.462.932 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.463.032 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.463.044 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.463.053 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.463.061 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.463.069 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.463.078 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.463.086 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.463.094 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.463.103 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.463.111 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [3] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.463.119 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [0] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.463.127 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.463.135 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.463.144 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.463.152 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.463.165 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:37:36.463.188 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.463.202 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.463.238 [graph_prepare.cc:1988][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [337] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.463.252 [graph_manager.cc:1065][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1249] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.475.283 [graph_manager.cc:1077][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12011] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.475.389 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.475.439 [graph_manager.cc:1080][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [120] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.483.657 [graph_manager.cc:1081][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [8200] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.483.712 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.483.728 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.483.741 [graph_manager.cc:1082][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.483.777 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.483.792 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.483.806 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.483.989 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [174] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.484.008 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.484.129 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [109] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.484.145 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.484.202 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [45] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.484.224 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.484.261 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.484.370 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [97] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.484.388 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.484.401 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.484.411 [graph_manager.cc:2700][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [641] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.484.702 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of EnterPass is [5] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:37:36.484.717 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:37:36.484.727 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [6] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:37:36.484.736 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:37:36.484.745 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:37:36.484.754 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CastRemovePass is [56] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:37:36.484.763 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:37:36.484.771 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [7] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:37:36.484.779 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [12] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:37:36.484.787 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [6] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:37:36.484.796 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [22] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:37:36.484.804 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [18] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:37:36.484.812 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [24] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:37:36.484.820 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [10] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:37:36.484.828 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [5] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:37:36.484.838 [graph_manager.cc:2741][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [409] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.484.847 [graph_manager.cc:2752][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.484.872 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.484.891 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.484.916 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.484.931 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.484.943 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.484.956 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.484.977 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.484.992 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.485.006 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.485.017 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.485.029 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.485.042 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.485.068 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.485.081 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.485.090 [graph_manager.cc:2810][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [224] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.485.142 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:37:36.485.154 [graph_manager.cc:2821][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [55] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.485.184 [graph_manager.cc:1087][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1422] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.485.909 [graph_manager.cc:1088][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [710] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.485.977 [graph_manager.cc:1089][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.486.000 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.486.019 [graph_manager.cc:1097][EVENT]167204 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:37:36.486.044 [graph_manager.cc:3325][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.496.397 [engine_place.cc:144][EVENT]167204 Run:The time cost of AIcoreEngine::CheckSupported is [10077] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.496.428 [engine_place.cc:144][EVENT]167204 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.496.438 [engine_place.cc:144][EVENT]167204 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.496.543 [graph_manager.cc:3351][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10477] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.496.561 [graph_manager.cc:3364][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.496.651 [engine_partitioner.cc:1139][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.496.685 [engine_partitioner.cc:1142][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.496.901 [engine_partitioner.cc:1148][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [205] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.496.948 [engine_partitioner.cc:1155][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [34] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.496.999 [engine_partitioner.cc:1164][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [40] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.497.036 [graph_manager.cc:3405][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [462] micro second. [INFO] GE(164039,python):2024-01-10-11:37:36.497.055 [graph_manager.cc:3412][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. - \ [INFO] GE(164039,python):2024-01-10-11:38:00.536.459 [graph_manager.cc:3422][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [24039388] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.536.538 [graph_manager.cc:3428][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.536.803 [graph_manager.cc:3467][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [238] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.536.825 [graph_manager.cc:3377][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [24040252] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.536.844 [graph_manager.cc:1106][EVENT]167204 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [24050809] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.536.857 [graph_manager.cc:1115][EVENT]167204 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:38:00.536.891 [graph_manager.cc:1130][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.536.925 [graph_manager.cc:1131][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.536.975 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.536.998 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.537.009 [graph_manager.cc:2837][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [51] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.537.206 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [40] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:00.537.220 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:00.537.229 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:00.537.238 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of BitcastPass is [5] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:00.537.247 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [11] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:00.537.256 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [23] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:00.537.266 [graph_manager.cc:2864][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [236] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.537.279 [graph_manager.cc:2872][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.537.300 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.537.316 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.537.333 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.537.348 [compile_nodes_pass.cc:88][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.537.358 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.537.368 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.537.503 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [125] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.537.562 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [47] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.537.578 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.537.593 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.537.610 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.537.625 [graph_manager.cc:2927][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [330] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.537.638 [graph_manager.cc:2937][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.537.656 [graph_manager.cc:2943][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.537.668 [graph_manager.cc:2950][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.537.959 [graph_manager.cc:2958][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [70] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.537.998 [graph_manager.cc:1132][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [1045] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.538.095 [graph_manager.cc:1135][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [82] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.538.140 [graph_manager.cc:2975][EVENT]167204 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [27] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.538.173 [graph_manager.cc:2981][EVENT]167204 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.538.189 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.538.199 [graph_manager.cc:2986][EVENT]167204 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.538.207 [graph_manager.cc:1136][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [96] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.538.604 [graph_manager.cc:3555][EVENT]167204 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [349] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.538.769 [engine_partitioner.cc:1139][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [43] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.538.807 [engine_partitioner.cc:1142][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.539.003 [engine_partitioner.cc:1148][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [185] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.539.046 [engine_partitioner.cc:1155][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [29] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.539.096 [engine_partitioner.cc:1164][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.539.125 [graph_builder.cc:865][EVENT]167204 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [436] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:38:00.539.582 [logger.cc:1071] 167204 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:38:00.539.624 [task_generator.cc:804][EVENT]167204 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [99] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.539.725 [task_generator.cc:805][EVENT]167204 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [88] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.541.875 [task_generator.cc:814][EVENT]167204 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [2135] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.541.891 [task_generator.cc:954][EVENT]167204 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2367] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.541.973 [task_generator.cc:967][EVENT]167204 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [44] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:38:00.541.999 [logger.cc:1084] 167204 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:38:00.543.246 [graph_manager.cc:1152][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [5001] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.543.282 [graph_manager.cc:1164][EVENT]167204 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:38:00.543.326 [graph_manager.cc:1271][EVENT]167204 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [24085809] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.543.338 [graph_manager.cc:1272][EVENT]167204 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:38:00.543.668 [atrace_api.c:93](tid:167204) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:38:00.543.687 [atrace_api.c:95](tid:167204) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:38:00.573.850 [graph_converter.cc:838][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [11503] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.574.103 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of ZeroCopy is [193] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.575.997 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CEM is [1868] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.576.502 [copy_flow_launch_fuse.cc:395][EVENT]167204 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [475] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.576.528 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [504] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.576.828 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [287] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.576.929 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [78] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.577.014 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of ZeroCopy is [64] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.577.580 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CEM is [549] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.577.869 [copy_flow_launch_fuse.cc:395][EVENT]167204 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [263] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.577.893 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [287] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.577.973 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [69] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.578.045 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [58] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.578.137 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of ZeroCopy is [62] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.578.410 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CEM is [259] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.578.648 [copy_flow_launch_fuse.cc:395][EVENT]167204 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [221] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.578.667 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [241] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.578.745 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [67] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.578.815 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [58] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.578.835 [graph_converter.cc:849][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4932] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.579.732 [graph_converter.cc:853][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [886] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.582.154 [graph_converter.cc:857][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2398] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.582.642 [graph_converter.cc:862][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [459] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.657.470 [graph_var_manager.cc:1424][EVENT]167201 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:38:00.657.561 [graph_manager.cc:1248][EVENT]167201 PreRun:PreRun start: graph node size 4, session id 22, graph id 21, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:38:00.657.843 [atrace_api.c:28](tid:167201) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:38:00.657.872 [trace_rb_log.c:84](tid:167201) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:38:00.657.885 [atrace_api.c:32](tid:167201) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:38:00.657.910 [client_manager.cpp:157][SetProfilingCallback][tid:167201] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:38:00.658.321 [parallel_partitioner.cc:165][EVENT]167201 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.658.358 [parallel_partitioner.cc:178][EVENT]167201 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.658.405 [graph_prepare.cc:1378][EVENT]167201 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.658.586 [graph_manager.cc:1050][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [198] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.658.610 [graph_manager.cc:1052][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.658.746 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.658.775 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.658.839 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.658.853 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.658.898 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.658.911 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.658.929 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.659.016 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.659.036 [graph_manager.cc:1054][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [414] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.659.272 [graph_manager.cc:1055][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [223] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.660.242 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:38:00.660.268 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.660.280 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.660.290 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [332] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.660.299 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [15] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.660.308 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:38:00.660.317 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.660.326 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.660.334 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.662.123 [graph_manager.cc:1056][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2833] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.662.190 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.662.208 [graph_prepare.cc:1982][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [52] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.662.679 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:38:00.662.704 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.662.725 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.662.735 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [260] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.662.744 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.662.753 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:38:00.662.761 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [9] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.662.770 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.662.778 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.662.825 [graph_prepare.cc:1983][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [604] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.662.849 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.662.861 [graph_prepare.cc:1984][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.662.875 [graph_prepare.cc:1985][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.662.889 [graph_prepare.cc:1986][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.662.900 [graph_prepare.cc:1987][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.662.914 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.662.925 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.662.937 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.663.027 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.663.038 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.663.047 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.663.056 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.663.064 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.663.073 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.663.081 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.663.095 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.663.104 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.663.113 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.663.121 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.663.129 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.663.137 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.663.145 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.663.154 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.663.162 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.663.185 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.663.197 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.663.228 [graph_prepare.cc:1988][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [320] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.663.242 [graph_manager.cc:1065][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1086] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.675.084 [graph_manager.cc:1077][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11823] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.675.155 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.675.203 [graph_manager.cc:1080][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [85] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.679.762 [graph_manager.cc:1081][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [4543] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.679.804 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.679.819 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.679.831 [graph_manager.cc:1082][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.679.863 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.679.877 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.679.904 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.679.938 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.679.952 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.679.966 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.679.978 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.018 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.037 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.056 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.085 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.100 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.112 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.122 [graph_manager.cc:2700][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [265] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.247 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.680.260 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.680.269 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.680.278 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.680.287 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.680.295 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.680.303 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.680.312 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.680.320 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.680.328 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.680.343 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.680.352 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.680.360 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.680.369 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.680.377 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.680.386 [graph_manager.cc:2741][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [246] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.395 [graph_manager.cc:2752][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.417 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.428 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.444 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.459 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.471 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.483 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.502 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.515 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.528 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.539 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.551 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.562 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.581 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.593 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.602 [graph_manager.cc:2810][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [189] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.631 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.680.646 [graph_manager.cc:2821][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.675 [graph_manager.cc:1087][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [824] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.806 [graph_manager.cc:1088][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [119] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.846 [graph_manager.cc:1089][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.863 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.680.878 [graph_manager.cc:1097][EVENT]167201 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:38:00.680.899 [graph_manager.cc:3325][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.681.374 [engine_place.cc:144][EVENT]167201 Run:The time cost of AIcoreEngine::CheckSupported is [378] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.681.399 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.681.408 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.681.487 [graph_manager.cc:3351][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [575] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.681.505 [graph_manager.cc:3364][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.681.571 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.681.588 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.681.766 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [168] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.681.811 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [30] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.681.858 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.681.891 [graph_manager.cc:3405][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [373] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.681.909 [graph_manager.cc:3412][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.163 [graph_manager.cc:3422][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [16238] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.203 [graph_manager.cc:3428][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.348 [graph_manager.cc:3467][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [117] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.367 [graph_manager.cc:3377][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [16850] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.382 [graph_manager.cc:1106][EVENT]167201 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [17490] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.395 [graph_manager.cc:1115][EVENT]167201 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:38:00.698.419 [graph_manager.cc:1130][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.452 [graph_manager.cc:1131][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.477 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.495 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.505 [graph_manager.cc:2837][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.590 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.698.602 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.698.611 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.698.620 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.698.628 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.698.637 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.698.647 [graph_manager.cc:2864][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [125] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.658 [graph_manager.cc:2872][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.676 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.690 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.705 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.718 [compile_nodes_pass.cc:88][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.734 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.744 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.825 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [71] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.852 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.865 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.878 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.890 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.899 [graph_manager.cc:2927][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [227] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.911 [graph_manager.cc:2937][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.925 [graph_manager.cc:2943][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.698.936 [graph_manager.cc:2950][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.699.121 [graph_manager.cc:2958][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.699.153 [graph_manager.cc:1132][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [687] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.699.222 [graph_manager.cc:1135][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [56] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.699.256 [graph_manager.cc:2975][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.699.289 [graph_manager.cc:2981][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.699.304 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.699.314 [graph_manager.cc:2986][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.699.323 [graph_manager.cc:1136][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [84] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.699.442 [graph_manager.cc:3555][EVENT]167201 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [86] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.699.535 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.699.561 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.699.682 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [110] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.699.715 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.699.755 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [29] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.699.777 [graph_builder.cc:865][EVENT]167201 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [277] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:38:00.700.063 [logger.cc:1071] 167201 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:38:00.700.092 [task_generator.cc:804][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [72] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.700.153 [task_generator.cc:805][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [48] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.700.824 [task_generator.cc:814][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [658] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.700.837 [task_generator.cc:954][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [819] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.700.897 [task_generator.cc:967][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [33] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:38:00.700.916 [logger.cc:1084] 167201 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:38:00.701.650 [graph_manager.cc:1152][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2301] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.701.680 [graph_manager.cc:1164][EVENT]167201 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:38:00.701.737 [graph_manager.cc:1271][EVENT]167201 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [43507] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.701.750 [graph_manager.cc:1272][EVENT]167201 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:38:00.702.065 [atrace_api.c:93](tid:167201) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:38:00.702.083 [atrace_api.c:95](tid:167201) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:38:00.715.239 [graph_converter.cc:838][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [4392] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.715.409 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [122] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.715.895 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [466] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.716.103 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [185] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.716.123 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [206] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.716.359 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [211] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.716.398 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.716.428 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.716.621 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [181] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.716.704 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [66] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.716.718 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [80] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.716.747 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.716.772 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.716.798 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.716.873 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [66] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.716.939 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [56] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.716.950 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [67] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.716.976 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.716.999 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.717.011 [graph_converter.cc:849][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1731] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.717.234 [graph_converter.cc:853][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [213] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.717.956 [graph_converter.cc:857][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [707] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.718.104 [graph_converter.cc:862][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [123] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.785.304 [graph_var_manager.cc:1424][EVENT]167204 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:38:00.785.395 [graph_manager.cc:1248][EVENT]167204 PreRun:PreRun start: graph node size 4, session id 23, graph id 22, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:38:00.785.675 [atrace_api.c:28](tid:167204) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:38:00.785.725 [trace_rb_log.c:84](tid:167204) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:38:00.785.740 [atrace_api.c:32](tid:167204) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:38:00.785.759 [client_manager.cpp:157][SetProfilingCallback][tid:167204] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:38:00.786.201 [parallel_partitioner.cc:165][EVENT]167204 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.786.237 [parallel_partitioner.cc:178][EVENT]167204 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.786.284 [graph_prepare.cc:1378][EVENT]167204 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.786.488 [graph_manager.cc:1050][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [222] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.786.513 [graph_manager.cc:1052][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.786.656 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.786.687 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.786.741 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [42] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.786.754 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.786.805 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.786.818 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.786.835 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.786.920 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.786.941 [graph_manager.cc:1054][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [415] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.787.180 [graph_manager.cc:1055][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [227] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.788.183 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:38:00.788.209 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.788.220 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.788.230 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferShapePass is [321] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.788.239 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.788.248 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:38:00.788.256 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [10] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.788.273 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.788.283 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.023 [graph_manager.cc:1056][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2823] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.790.090 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.109 [graph_prepare.cc:1982][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.790.540 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:38:00.790.563 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.575 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.584 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferShapePass is [232] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.593 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.602 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:38:00.790.610 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.619 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.627 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.654 [graph_prepare.cc:1983][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [531] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.790.677 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.790.688 [graph_prepare.cc:1984][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.790.702 [graph_prepare.cc:1985][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.790.718 [graph_prepare.cc:1986][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.790.728 [graph_prepare.cc:1987][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.790.742 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.790.753 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.790.774 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.790.871 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.883 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.892 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.901 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.909 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.918 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.926 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.934 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.943 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.951 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.959 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.967 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.975 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.983 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.790.992 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.791.000 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.791.023 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.791.036 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.791.068 [graph_prepare.cc:1988][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [330] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.791.081 [graph_manager.cc:1065][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1026] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.803.583 [graph_manager.cc:1077][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12483] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.803.646 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.803.706 [graph_manager.cc:1080][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [88] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.806.836 [graph_manager.cc:1081][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3114] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.806.877 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.806.893 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.806.904 [graph_manager.cc:1082][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.806.935 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.806.950 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.806.964 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.806.991 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.004 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.019 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.031 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.072 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.089 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.119 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.146 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.160 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.172 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.181 [graph_manager.cc:2700][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [250] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.303 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.807.316 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.807.326 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.807.348 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.807.357 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.807.366 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.807.375 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.807.383 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.807.391 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.807.400 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.807.408 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.807.416 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.807.424 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.807.432 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.807.440 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.807.450 [graph_manager.cc:2741][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [251] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.460 [graph_manager.cc:2752][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.483 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.494 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.511 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.527 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.539 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.552 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.571 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.584 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.596 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.611 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.630 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.642 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.661 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.673 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.682 [graph_manager.cc:2810][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [203] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.712 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.807.723 [graph_manager.cc:2821][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [33] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.753 [graph_manager.cc:1087][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [830] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.884 [graph_manager.cc:1088][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [119] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.924 [graph_manager.cc:1089][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.941 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.807.970 [graph_manager.cc:1097][EVENT]167204 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:38:00.807.992 [graph_manager.cc:3325][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.808.359 [engine_place.cc:144][EVENT]167204 Run:The time cost of AIcoreEngine::CheckSupported is [263] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.808.383 [engine_place.cc:144][EVENT]167204 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.808.392 [engine_place.cc:144][EVENT]167204 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.808.478 [graph_manager.cc:3351][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [472] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.808.497 [graph_manager.cc:3364][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.808.574 [engine_partitioner.cc:1139][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.808.593 [engine_partitioner.cc:1142][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.808.757 [engine_partitioner.cc:1148][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [147] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.808.800 [engine_partitioner.cc:1155][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [29] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.808.847 [engine_partitioner.cc:1164][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.808.881 [graph_manager.cc:3405][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [372] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.808.899 [graph_manager.cc:3412][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.974.470 [graph_manager.cc:3422][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [165556] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.974.518 [graph_manager.cc:3428][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.974.672 [graph_manager.cc:3467][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [131] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.974.691 [graph_manager.cc:3377][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [166182] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.974.707 [graph_manager.cc:1106][EVENT]167204 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [166721] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.974.719 [graph_manager.cc:1115][EVENT]167204 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:38:00.974.743 [graph_manager.cc:1130][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.974.778 [graph_manager.cc:1131][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.974.805 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.974.822 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.974.832 [graph_manager.cc:2837][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.974.922 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.974.935 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.974.944 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.974.952 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.974.961 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.974.981 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:38:00.974.992 [graph_manager.cc:2864][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [142] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.004 [graph_manager.cc:2872][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.023 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.036 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.052 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.066 [compile_nodes_pass.cc:88][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.076 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.087 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.173 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [76] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.203 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.215 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.229 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.243 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.251 [graph_manager.cc:2927][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [232] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.265 [graph_manager.cc:2937][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.294 [graph_manager.cc:2943][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.308 [graph_manager.cc:2950][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.509 [graph_manager.cc:2958][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.540 [graph_manager.cc:1132][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [747] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.642 [graph_manager.cc:1135][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [88] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.688 [graph_manager.cc:2975][EVENT]167204 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.815 [graph_manager.cc:2981][EVENT]167204 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [114] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.831 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.841 [graph_manager.cc:2986][EVENT]167204 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.850 [graph_manager.cc:1136][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [184] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.975.968 [graph_manager.cc:3555][EVENT]167204 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [89] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.976.036 [engine_partitioner.cc:1139][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.976.053 [engine_partitioner.cc:1142][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.976.181 [engine_partitioner.cc:1148][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [118] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.976.215 [engine_partitioner.cc:1155][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.976.254 [engine_partitioner.cc:1164][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [29] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.976.279 [graph_builder.cc:865][EVENT]167204 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [280] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.976.361 [graph_builder.cc:288][EVENT]167204 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [65] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.976.476 [graph_builder.cc:293][EVENT]167204 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [102] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.976.674 [model_builder.cc:1133][EVENT]167204 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [102] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.976.951 [block_mem_assigner.cc:4069][EVENT]169349 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164039,python):2024-01-10-11:38:00.976.953 [block_mem_assigner.cc:4069][EVENT]169350 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164039,python):2024-01-10-11:38:00.977.373 [graph_mem_assigner.cc:2166][EVENT]167204 SetInputOffset:[IMAS]AfterAssignMemory : online_22 memoffset[1024], memtype[2], theory_min[2048], zero_copy[1024], total_size[1024], no_reuse[1024], streams[1], topo_mode[DFS], mop[], io_reuse[0:0], alloc_mode[] [INFO] GE(164039,python):2024-01-10-11:38:00.977.471 [model_builder.cc:1144][EVENT]167204 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [776] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.977.498 [model_builder.cc:1152][EVENT]167204 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.977.521 [model_builder.cc:1157][EVENT]167204 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.977.648 [model_builder.cc:1167][EVENT]167204 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [115] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.977.666 [model_builder.cc:1174][EVENT]167204 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.977.714 [model_builder.cc:1180][EVENT]167204 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [33] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.977.754 [model_builder.cc:1184][EVENT]167204 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [25] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.977.774 [graph_builder.cc:304][EVENT]167204 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [1276] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:38:00.977.888 [logger.cc:1071] 167204 ModelBindStream: model_id=832, stream_id=65, flag=0. [INFO] GE(164039,python):2024-01-10-11:38:00.977.993 [task_generator.cc:804][EVENT]167204 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.978.064 [task_generator.cc:805][EVENT]167204 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [56] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.978.819 [task_generator.cc:814][EVENT]167204 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [741] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.978.833 [task_generator.cc:954][EVENT]167204 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [845] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.978.895 [task_generator.cc:967][EVENT]167204 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [34] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:38:00.978.915 [logger.cc:1084] 167204 ModelUnbindStream: model_id=832, stream_id=65, [INFO] GE(164039,python):2024-01-10-11:38:00.978.974 [graph_builder.cc:310][EVENT]167204 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [1187] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.979.101 [graph_manager.cc:1152][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3228] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.979.118 [graph_manager.cc:1164][EVENT]167204 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:38:00.979.151 [graph_manager.cc:1271][EVENT]167204 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [193042] micro second. [INFO] GE(164039,python):2024-01-10-11:38:00.979.161 [graph_manager.cc:1272][EVENT]167204 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:38:00.979.479 [atrace_api.c:93](tid:167204) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:38:00.979.495 [atrace_api.c:95](tid:167204) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:38:00.980.132 [model_introduction.cc:236][EVENT]167204 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(164039,python):2024-01-10-11:38:00.980.153 [model_introduction.cc:294][EVENT]167204 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(164039,python):2024-01-10-11:38:00.980.166 [model_introduction.cc:366][EVENT]167204 Data:model io_info size:222 [INFO] GE(164039,python):2024-01-10-11:38:01.044.808 [graph_var_manager.cc:1424][EVENT]167204 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:38:01.044.913 [graph_manager.cc:1248][EVENT]167204 PreRun:PreRun start: graph node size 5, session id 24, graph id 23, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:38:01.045.164 [atrace_api.c:28](tid:167204) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:38:01.045.196 [trace_rb_log.c:84](tid:167204) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:38:01.045.208 [atrace_api.c:32](tid:167204) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:38:01.045.228 [client_manager.cpp:157][SetProfilingCallback][tid:167204] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:38:01.045.636 [parallel_partitioner.cc:165][EVENT]167204 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.045.675 [parallel_partitioner.cc:178][EVENT]167204 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.045.740 [graph_prepare.cc:1378][EVENT]167204 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.045.898 [graph_manager.cc:1050][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [176] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.045.922 [graph_manager.cc:1052][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.046.093 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.046.123 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.046.171 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.046.184 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.046.231 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.046.244 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.046.261 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.046.350 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.046.371 [graph_manager.cc:1054][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [436] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.046.608 [graph_manager.cc:1055][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [225] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.047.912 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.047.938 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.047.950 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.047.969 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferShapePass is [439] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.047.979 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [15] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.047.989 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.047.997 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.048.006 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.048.014 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.051.952 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.051.982 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.051.994 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.052.004 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferShapePass is [363] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.052.013 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.052.022 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.052.030 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.052.039 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.052.047 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.053.327 [graph_manager.cc:1056][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [6698] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.053.396 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.053.415 [graph_prepare.cc:1982][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [57] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.054.055 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.054.081 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.092 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.102 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferShapePass is [350] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.111 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.130 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.054.140 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [23] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.149 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.157 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.212 [graph_prepare.cc:1983][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [782] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.054.238 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.054.250 [graph_prepare.cc:1984][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.054.265 [graph_prepare.cc:1985][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.054.279 [graph_prepare.cc:1986][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.054.290 [graph_prepare.cc:1987][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.054.304 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.054.315 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.054.328 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.054.429 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.441 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.451 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.459 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.468 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.476 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.485 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.493 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.501 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.509 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.524 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.533 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.542 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.550 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [7] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.558 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.566 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.054.590 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.054.602 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.054.639 [graph_prepare.cc:1988][EVENT]167204 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [341] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.054.653 [graph_manager.cc:1065][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1295] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.067.178 [graph_manager.cc:1077][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12506] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.067.291 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.067.342 [graph_manager.cc:1080][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [126] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.072.499 [graph_manager.cc:1081][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [5140] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.072.541 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.072.556 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.072.567 [graph_manager.cc:1082][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.072.599 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.072.612 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.072.627 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.072.790 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [154] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.072.820 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.072.927 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [96] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.072.944 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.072.999 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [45] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.022 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.051 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.148 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [87] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.166 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.179 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.188 [graph_manager.cc:2700][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [595] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.453 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.073.469 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AddNPass is [4] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.073.479 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.073.488 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [4] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.073.497 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.073.505 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CastRemovePass is [45] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.073.513 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [5] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.073.522 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.073.530 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [11] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.073.538 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [4] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.073.546 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [18] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.073.555 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.073.563 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [23] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.073.579 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [9] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.073.588 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [6] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.073.598 [graph_manager.cc:2741][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [391] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.607 [graph_manager.cc:2752][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.632 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.644 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.667 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.681 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.713 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.733 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.757 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.771 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.785 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.796 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.810 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.822 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.846 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.860 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.869 [graph_manager.cc:2810][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [242] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.918 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.073.930 [graph_manager.cc:2821][EVENT]167204 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [52] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.073.959 [graph_manager.cc:1087][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1373] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.074.561 [graph_manager.cc:1088][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [582] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.074.625 [graph_manager.cc:1089][EVENT]167204 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.074.648 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.074.666 [graph_manager.cc:1097][EVENT]167204 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:38:01.074.689 [graph_manager.cc:3325][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.084.276 [engine_place.cc:144][EVENT]167204 Run:The time cost of AIcoreEngine::CheckSupported is [9346] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.084.309 [engine_place.cc:144][EVENT]167204 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.084.320 [engine_place.cc:144][EVENT]167204 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.084.426 [graph_manager.cc:3351][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [9723] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.084.445 [graph_manager.cc:3364][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.084.536 [engine_partitioner.cc:1139][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [32] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.084.569 [engine_partitioner.cc:1142][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.084.775 [engine_partitioner.cc:1148][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [196] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.084.821 [engine_partitioner.cc:1155][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [32] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.084.870 [engine_partitioner.cc:1164][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.084.908 [graph_manager.cc:3405][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [449] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.084.928 [graph_manager.cc:3412][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.368.624 [graph_manager.cc:3422][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [283678] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.368.703 [graph_manager.cc:3428][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.368.955 [graph_manager.cc:3467][EVENT]167204 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [226] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.368.974 [graph_manager.cc:3377][EVENT]167204 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [284517] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.007 [graph_manager.cc:1106][EVENT]167204 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [294325] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.020 [graph_manager.cc:1115][EVENT]167204 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:38:01.369.050 [graph_manager.cc:1130][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.085 [graph_manager.cc:1131][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.120 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.144 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.154 [graph_manager.cc:2837][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [51] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.342 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [39] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.369.357 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.369.367 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.369.376 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of BitcastPass is [0] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.369.385 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [12] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.369.394 [base_pass.cc:339][EVENT]167204 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [22] micro second, call num is [11] [INFO] GE(164039,python):2024-01-10-11:38:01.369.405 [graph_manager.cc:2864][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [231] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.418 [graph_manager.cc:2872][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.440 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.455 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.473 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.488 [compile_nodes_pass.cc:88][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.498 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.508 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.651 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [117] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.730 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [65] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.746 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.762 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.778 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.788 [graph_manager.cc:2927][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [354] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.800 [graph_manager.cc:2937][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.816 [graph_manager.cc:2943][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.369.827 [graph_manager.cc:2950][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.370.065 [graph_manager.cc:2958][EVENT]167204 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [63] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.370.103 [graph_manager.cc:1132][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [1003] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.370.198 [graph_manager.cc:1135][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [81] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.370.241 [graph_manager.cc:2975][EVENT]167204 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [26] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.370.272 [graph_manager.cc:2981][EVENT]167204 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.370.287 [pass_manager.cc:82][EVENT]167204 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.370.297 [graph_manager.cc:2986][EVENT]167204 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.370.306 [graph_manager.cc:1136][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [93] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.370.689 [graph_manager.cc:3555][EVENT]167204 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [339] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.370.843 [engine_partitioner.cc:1139][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [40] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.370.879 [engine_partitioner.cc:1142][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.371.071 [engine_partitioner.cc:1148][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [181] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.371.122 [engine_partitioner.cc:1155][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [29] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.371.172 [engine_partitioner.cc:1164][EVENT]167204 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.371.200 [graph_builder.cc:865][EVENT]167204 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [433] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:38:01.371.655 [logger.cc:1071] 167204 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:38:01.371.688 [task_generator.cc:804][EVENT]167204 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [89] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.371.783 [task_generator.cc:805][EVENT]167204 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [82] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.373.715 [task_generator.cc:814][EVENT]167204 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1917] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.373.733 [task_generator.cc:954][EVENT]167204 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2134] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.373.811 [task_generator.cc:967][EVENT]167204 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [43] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:38:01.373.836 [logger.cc:1084] 167204 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:38:01.422.573 [graph_manager.cc:1152][EVENT]167204 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [52229] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.422.649 [graph_manager.cc:1164][EVENT]167204 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:38:01.422.704 [graph_manager.cc:1271][EVENT]167204 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [377163] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.422.717 [graph_manager.cc:1272][EVENT]167204 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:38:01.423.064 [atrace_api.c:93](tid:167204) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:38:01.423.082 [atrace_api.c:95](tid:167204) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:38:01.615.260 [graph_converter.cc:838][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [66181] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.615.576 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of ZeroCopy is [234] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.617.407 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CEM is [1805] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.617.919 [copy_flow_launch_fuse.cc:395][EVENT]167204 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [484] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.617.947 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [513] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.618.240 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [280] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.618.334 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [71] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.618.412 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of ZeroCopy is [60] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.618.997 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CEM is [552] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.619.246 [copy_flow_launch_fuse.cc:395][EVENT]167204 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [221] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.619.268 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [245] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.619.344 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [64] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.619.413 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [55] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.619.480 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of ZeroCopy is [56] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.619.747 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CEM is [254] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.619.973 [copy_flow_launch_fuse.cc:395][EVENT]167204 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [208] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.619.992 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [228] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.620.063 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [62] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.620.129 [base_optimizer.cc:70][EVENT]167204 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [53] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.620.149 [graph_converter.cc:849][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4812] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.621.044 [graph_converter.cc:853][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [884] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.623.447 [graph_converter.cc:857][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2380] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.623.914 [graph_converter.cc:862][EVENT]167204 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [437] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.763.370 [graph_var_manager.cc:1424][EVENT]167201 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:38:01.763.456 [graph_manager.cc:1248][EVENT]167201 PreRun:PreRun start: graph node size 5, session id 25, graph id 24, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:38:01.763.695 [atrace_api.c:28](tid:167201) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:38:01.763.726 [trace_rb_log.c:84](tid:167201) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:38:01.763.739 [atrace_api.c:32](tid:167201) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:38:01.763.757 [client_manager.cpp:157][SetProfilingCallback][tid:167201] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:38:01.764.160 [parallel_partitioner.cc:165][EVENT]167201 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.764.198 [parallel_partitioner.cc:178][EVENT]167201 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.764.262 [graph_prepare.cc:1378][EVENT]167201 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.764.427 [graph_manager.cc:1050][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [183] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.764.452 [graph_manager.cc:1052][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.764.620 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.764.652 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.764.701 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.764.714 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.764.772 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.764.785 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.764.803 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.764.896 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [3] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.764.916 [graph_manager.cc:1054][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [452] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.765.154 [graph_manager.cc:1055][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [225] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.766.494 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.766.521 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.766.532 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.766.542 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [439] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.766.551 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.766.560 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [4] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.766.568 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.766.577 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.766.585 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [7] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.770.653 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.770.694 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.770.707 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.770.717 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [332] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.770.726 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.770.735 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.770.743 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.770.752 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.770.760 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.772.049 [graph_manager.cc:1056][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [6874] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.772.116 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [6] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.772.133 [graph_prepare.cc:1982][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.772.724 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.772.748 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.772.759 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.772.769 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [326] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.772.778 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.772.787 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.772.795 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.772.803 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.772.812 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.772.864 [graph_prepare.cc:1983][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [717] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.772.890 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.772.909 [graph_prepare.cc:1984][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [28] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.772.924 [graph_prepare.cc:1985][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.772.939 [graph_prepare.cc:1986][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.772.951 [graph_prepare.cc:1987][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.772.967 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.772.978 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.772.992 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.773.093 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.773.104 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.773.113 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.773.122 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.773.130 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.773.139 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.773.147 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.773.155 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.773.164 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.773.172 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.773.180 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.773.189 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SnapshotPass is [3] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.773.197 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [4] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.773.205 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [7] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.773.213 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.773.222 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [6] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:01.773.249 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.773.262 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.773.298 [graph_prepare.cc:1988][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [336] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.773.311 [graph_manager.cc:1065][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1232] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.786.842 [graph_manager.cc:1077][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13512] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.786.917 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.786.966 [graph_manager.cc:1080][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [88] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.792.019 [graph_manager.cc:1081][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [5038] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.792.063 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.792.080 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.792.092 [graph_manager.cc:1082][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.792.125 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.792.139 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.792.154 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.792.317 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [154] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.792.336 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.792.428 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [82] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.792.445 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.792.498 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [41] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.792.520 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.792.549 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.792.655 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [80] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.792.673 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.792.686 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.792.696 [graph_manager.cc:2700][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [577] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.792.946 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [4] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.792.961 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.792.970 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.792.980 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.792.988 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.792.997 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CastRemovePass is [42] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.793.005 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.793.014 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.793.022 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [10] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.793.030 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.793.039 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [18] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.793.047 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [16] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.793.055 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.793.063 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [10] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.793.072 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.793.081 [graph_manager.cc:2741][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [367] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.793.090 [graph_manager.cc:2752][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.793.114 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.793.127 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.793.156 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.793.172 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.793.185 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.793.197 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.793.218 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.793.232 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.793.246 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.793.256 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.793.269 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.793.281 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.793.304 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.793.317 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.793.326 [graph_manager.cc:2810][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [217] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.793.373 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.793.384 [graph_manager.cc:2821][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [50] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.793.413 [graph_manager.cc:1087][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1302] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.794.084 [graph_manager.cc:1088][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [658] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.794.147 [graph_manager.cc:1089][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [34] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.794.170 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.794.188 [graph_manager.cc:1097][EVENT]167201 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:38:01.794.210 [graph_manager.cc:3325][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.801.965 [engine_place.cc:144][EVENT]167201 Run:The time cost of AIcoreEngine::CheckSupported is [7480] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.802.007 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.802.019 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.802.121 [graph_manager.cc:3351][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [7897] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.802.141 [graph_manager.cc:3364][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.802.223 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [30] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.802.255 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.802.456 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [190] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.802.502 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [32] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.802.552 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.802.589 [graph_manager.cc:3405][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [435] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.802.607 [graph_manager.cc:3412][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.908.303 [graph_manager.cc:3422][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [105682] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.908.347 [graph_manager.cc:3428][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.908.530 [graph_manager.cc:3467][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [160] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.908.548 [graph_manager.cc:3377][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [106396] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.908.565 [graph_manager.cc:1106][EVENT]167201 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [114362] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.908.577 [graph_manager.cc:1115][EVENT]167201 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:38:01.908.603 [graph_manager.cc:1130][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.908.636 [graph_manager.cc:1131][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.908.663 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.908.698 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.908.708 [graph_manager.cc:2837][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [56] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.908.854 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [28] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.908.867 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.908.876 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.908.885 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of BitcastPass is [3] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.908.894 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [7] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.908.902 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [17] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:01.908.913 [graph_manager.cc:2864][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [187] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.908.925 [graph_manager.cc:2872][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.908.945 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.908.960 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.908.977 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.908.991 [compile_nodes_pass.cc:88][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.909.000 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.909.011 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.909.125 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [106] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.909.176 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.909.190 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.909.205 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.909.219 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.909.228 [graph_manager.cc:2927][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [288] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.909.247 [graph_manager.cc:2937][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.909.264 [graph_manager.cc:2943][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.909.275 [graph_manager.cc:2950][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.909.544 [graph_manager.cc:2958][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [60] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.909.579 [graph_manager.cc:1132][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [930] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.909.657 [graph_manager.cc:1135][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [66] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.909.707 [graph_manager.cc:2975][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [34] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.909.741 [graph_manager.cc:2981][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.909.756 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.909.766 [graph_manager.cc:2986][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.909.776 [graph_manager.cc:1136][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [102] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.910.126 [graph_manager.cc:3555][EVENT]167201 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [306] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.910.262 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [33] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.910.292 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.910.462 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [159] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.910.501 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [26] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.910.545 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [32] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.910.572 [graph_builder.cc:865][EVENT]167201 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [375] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:38:01.910.988 [logger.cc:1071] 167201 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:38:01.911.021 [task_generator.cc:804][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [82] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.911.112 [task_generator.cc:805][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [77] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.912.762 [task_generator.cc:814][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1627] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.912.777 [task_generator.cc:954][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1838] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.912.851 [task_generator.cc:967][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [40] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:38:01.912.877 [logger.cc:1084] 167201 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:38:01.913.883 [graph_manager.cc:1152][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4072] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.913.919 [graph_manager.cc:1164][EVENT]167201 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:38:01.913.956 [graph_manager.cc:1271][EVENT]167201 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [149891] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.913.967 [graph_manager.cc:1272][EVENT]167201 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:38:01.914.284 [atrace_api.c:93](tid:167201) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:38:01.914.302 [atrace_api.c:95](tid:167201) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:38:01.939.074 [graph_converter.cc:838][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [9624] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.939.304 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [173] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.940.925 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [1594] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.941.387 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [434] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.941.412 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [461] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.941.681 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [256] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.941.784 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [66] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.941.854 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [54] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.942.371 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [502] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.942.598 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [202] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.942.618 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [223] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.942.687 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [57] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.942.748 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [48] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.942.809 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [49] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.943.048 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [227] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.943.280 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [190] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.943.298 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [209] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.943.364 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [55] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.943.424 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [48] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.943.442 [graph_converter.cc:849][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4318] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.944.210 [graph_converter.cc:853][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [756] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.946.307 [graph_converter.cc:857][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2076] micro second. [INFO] GE(164039,python):2024-01-10-11:38:01.946.726 [graph_converter.cc:862][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [388] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.030.558 [graph_var_manager.cc:1424][EVENT]167203 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:38:02.030.645 [graph_manager.cc:1248][EVENT]167203 PreRun:PreRun start: graph node size 7, session id 26, graph id 25, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:38:02.030.907 [atrace_api.c:28](tid:167203) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:38:02.030.939 [trace_rb_log.c:84](tid:167203) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:38:02.030.952 [atrace_api.c:32](tid:167203) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:38:02.030.970 [client_manager.cpp:157][SetProfilingCallback][tid:167203] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:38:02.031.391 [parallel_partitioner.cc:165][EVENT]167203 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.031.430 [parallel_partitioner.cc:178][EVENT]167203 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.031.480 [graph_prepare.cc:1378][EVENT]167203 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.031.666 [graph_manager.cc:1050][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [207] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.031.691 [graph_manager.cc:1052][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.031.872 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.031.902 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.031.951 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.031.982 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.032.028 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.032.041 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.032.059 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.032.155 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.032.175 [graph_manager.cc:1054][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [471] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.032.410 [graph_manager.cc:1055][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [221] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.033.831 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [14] [INFO] GE(164039,python):2024-01-10-11:38:02.033.859 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [6] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.033.871 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.033.880 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferShapePass is [524] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.033.890 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.033.898 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [14] [INFO] GE(164039,python):2024-01-10-11:38:02.033.907 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [15] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.033.916 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [23] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.033.924 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.035.286 [graph_manager.cc:1056][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2855] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.035.359 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.035.375 [graph_prepare.cc:1982][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [59] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.036.029 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [14] [INFO] GE(164039,python):2024-01-10-11:38:02.036.053 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.064 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.074 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferShapePass is [407] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.091 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [15] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.101 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [14] [INFO] GE(164039,python):2024-01-10-11:38:02.036.110 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.118 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.127 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.155 [graph_prepare.cc:1983][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [768] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.036.179 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.036.190 [graph_prepare.cc:1984][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.036.204 [graph_prepare.cc:1985][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.036.219 [graph_prepare.cc:1986][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.036.229 [graph_prepare.cc:1987][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.036.244 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.036.255 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.036.268 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.036.386 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.397 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.406 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.415 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.423 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.432 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.440 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.448 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.471 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.480 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.489 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.497 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SnapshotPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.505 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.513 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.522 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.530 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.036.556 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.036.569 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.036.609 [graph_prepare.cc:1988][EVENT]167203 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [371] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.036.622 [graph_manager.cc:1065][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1306] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.049.371 [graph_manager.cc:1077][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12730] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.049.452 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.049.504 [graph_manager.cc:1080][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [96] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.053.314 [graph_manager.cc:1081][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3795] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.053.355 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.053.371 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.053.382 [graph_manager.cc:1082][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.053.417 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.053.432 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.053.447 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.053.493 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [24] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.053.507 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.053.524 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.053.537 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.053.586 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [40] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.053.605 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.053.635 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.053.668 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.053.683 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.053.716 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.053.726 [graph_manager.cc:2700][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [315] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.053.882 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.053.896 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.053.906 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.053.915 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.053.923 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.053.932 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CastRemovePass is [11] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.053.940 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.053.948 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.053.957 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.053.965 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.053.973 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.053.981 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.053.996 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.054.005 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.054.013 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [5] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.054.023 [graph_manager.cc:2741][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [277] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.032 [graph_manager.cc:2752][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.054 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.066 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.086 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.101 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.112 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.124 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.142 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.156 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.168 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.178 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.191 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.202 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.223 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.237 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.246 [graph_manager.cc:2810][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [196] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.283 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.054.294 [graph_manager.cc:2821][EVENT]167203 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.326 [graph_manager.cc:1087][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [921] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.458 [graph_manager.cc:1088][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [118] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.503 [graph_manager.cc:1089][EVENT]167203 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [26] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.521 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.054.562 [graph_manager.cc:1097][EVENT]167203 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:38:02.054.584 [graph_manager.cc:3325][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.055.077 [engine_place.cc:144][EVENT]167203 Run:The time cost of AIcoreEngine::CheckSupported is [375] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.055.101 [engine_place.cc:144][EVENT]167203 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.055.111 [engine_place.cc:144][EVENT]167203 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.055.201 [graph_manager.cc:3351][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [603] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.055.218 [graph_manager.cc:3364][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.055.288 [engine_partitioner.cc:1139][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [24] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.055.306 [engine_partitioner.cc:1142][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.055.546 [engine_partitioner.cc:1148][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [230] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.055.597 [engine_partitioner.cc:1155][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.055.648 [engine_partitioner.cc:1164][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.055.684 [graph_manager.cc:3405][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [454] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.055.703 [graph_manager.cc:3412][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. | [INFO] GE(164039,python):2024-01-10-11:38:02.066.530 [graph_manager.cc:3422][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [10813] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.066.576 [graph_manager.cc:3428][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.066.745 [graph_manager.cc:3467][EVENT]167203 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [148] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.066.778 [graph_manager.cc:3377][EVENT]167203 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [11548] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.066.795 [graph_manager.cc:1106][EVENT]167203 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [12218] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.066.808 [graph_manager.cc:1115][EVENT]167203 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:38:02.066.834 [graph_manager.cc:1130][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.066.869 [graph_manager.cc:1131][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.066.896 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.066.914 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.066.924 [graph_manager.cc:2837][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.028 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [21] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.067.041 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.067.050 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.067.060 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.067.069 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.067.077 [base_pass.cc:339][EVENT]167203 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:38:02.067.088 [graph_manager.cc:2864][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [146] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.100 [graph_manager.cc:2872][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.119 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.133 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.149 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.163 [compile_nodes_pass.cc:88][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.173 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.189 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.302 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [104] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.335 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.348 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.363 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.376 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.385 [graph_manager.cc:2927][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [269] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.397 [graph_manager.cc:2937][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.412 [graph_manager.cc:2943][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.423 [graph_manager.cc:2950][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.615 [graph_manager.cc:2958][EVENT]167203 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [51] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.646 [graph_manager.cc:1132][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [763] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.716 [graph_manager.cc:1135][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [57] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.758 [graph_manager.cc:2975][EVENT]167203 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [25] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.790 [graph_manager.cc:2981][EVENT]167203 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.805 [pass_manager.cc:82][EVENT]167203 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.815 [graph_manager.cc:2986][EVENT]167203 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.825 [graph_manager.cc:1136][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [94] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.067.968 [graph_manager.cc:3555][EVENT]167203 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [105] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.068.079 [engine_partitioner.cc:1139][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.068.096 [engine_partitioner.cc:1142][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.068.294 [engine_partitioner.cc:1148][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [180] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.068.335 [engine_partitioner.cc:1155][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [27] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.068.378 [engine_partitioner.cc:1164][EVENT]167203 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [32] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.068.402 [graph_builder.cc:865][EVENT]167203 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [370] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:38:02.068.734 [logger.cc:1071] 167203 ModelBindStream: model_id=1344, stream_id=1601, flag=0. [INFO] GE(164039,python):2024-01-10-11:38:02.068.764 [task_generator.cc:804][EVENT]167203 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [73] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.068.842 [task_generator.cc:805][EVENT]167203 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [65] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.069.491 [task_generator.cc:814][EVENT]167203 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [635] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.069.505 [task_generator.cc:954][EVENT]167203 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [814] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.069.563 [task_generator.cc:967][EVENT]167203 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [34] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:38:02.069.582 [logger.cc:1084] 167203 ModelUnbindStream: model_id=1344, stream_id=1601, [INFO] GE(164039,python):2024-01-10-11:38:02.069.776 [graph_manager.cc:1152][EVENT]167203 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [1921] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.069.796 [graph_manager.cc:1164][EVENT]167203 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:38:02.069.829 [graph_manager.cc:1271][EVENT]167203 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [38536] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.069.839 [graph_manager.cc:1272][EVENT]167203 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:38:02.070.147 [atrace_api.c:93](tid:167203) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:38:02.070.164 [atrace_api.c:95](tid:167203) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:38:02.077.307 [graph_converter.cc:838][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1755] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.077.392 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of ZeroCopy is [40] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.078.039 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CEM is [629] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.078.314 [copy_flow_launch_fuse.cc:395][EVENT]167203 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [248] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.078.335 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [271] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.078.391 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [44] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.078.435 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.078.469 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.078.674 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CEM is [194] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.078.778 [copy_flow_launch_fuse.cc:395][EVENT]167203 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [86] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.078.792 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [101] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.078.830 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [28] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.078.859 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.078.890 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.078.995 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CEM is [95] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.079.087 [copy_flow_launch_fuse.cc:395][EVENT]167203 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [80] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.079.099 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [92] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.079.133 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [26] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.079.162 [base_optimizer.cc:70][EVENT]167203 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.079.176 [graph_converter.cc:849][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1829] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.079.480 [graph_converter.cc:853][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [295] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.080.503 [graph_converter.cc:857][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1006] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.080.707 [graph_converter.cc:862][EVENT]167203 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [179] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.145.373 [graph_var_manager.cc:1424][EVENT]167201 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:38:02.145.460 [graph_manager.cc:1248][EVENT]167201 PreRun:PreRun start: graph node size 5, session id 27, graph id 26, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:38:02.145.739 [atrace_api.c:28](tid:167201) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:38:02.145.771 [trace_rb_log.c:84](tid:167201) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:38:02.145.785 [atrace_api.c:32](tid:167201) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:38:02.145.803 [client_manager.cpp:157][SetProfilingCallback][tid:167201] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:38:02.146.214 [parallel_partitioner.cc:165][EVENT]167201 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.146.269 [parallel_partitioner.cc:178][EVENT]167201 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.146.317 [graph_prepare.cc:1378][EVENT]167201 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.146.480 [graph_manager.cc:1050][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [182] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.146.505 [graph_manager.cc:1052][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.146.656 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [3] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.146.687 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.146.735 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.146.748 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.146.793 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.146.806 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.146.825 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.146.914 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [4] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.146.935 [graph_manager.cc:1054][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [418] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.147.165 [graph_manager.cc:1055][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [217] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.148.316 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:02.148.344 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.148.356 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.148.366 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [391] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.148.374 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.148.383 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:02.148.392 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.148.400 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [21] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.148.417 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.150.417 [graph_manager.cc:1056][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3232] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.150.486 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [6] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.150.504 [graph_prepare.cc:1982][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [55] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.151.122 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [5] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:02.151.145 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.157 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.166 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [349] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.175 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.184 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [5] micro second, call num is [10] [INFO] GE(164039,python):2024-01-10-11:38:02.151.192 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.201 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.209 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.262 [graph_prepare.cc:1983][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [744] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.151.287 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.151.298 [graph_prepare.cc:1984][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.151.312 [graph_prepare.cc:1985][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.151.327 [graph_prepare.cc:1986][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.151.338 [graph_prepare.cc:1987][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.151.352 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.151.363 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.151.377 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.151.478 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.498 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.507 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.516 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.525 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.533 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.542 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.550 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [3] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.558 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.566 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.575 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.583 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.591 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.599 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [7] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.608 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.616 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [5] [INFO] GE(164039,python):2024-01-10-11:38:02.151.640 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.151.653 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.151.688 [graph_prepare.cc:1988][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [341] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.151.701 [graph_manager.cc:1065][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1254] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.163.712 [graph_manager.cc:1077][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11991] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.163.789 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.163.839 [graph_manager.cc:1080][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [92] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.172.040 [graph_manager.cc:1081][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [8175] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.172.094 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.172.110 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.172.122 [graph_manager.cc:1082][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.172.156 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.172.171 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.172.186 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.172.374 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [179] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.172.392 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.172.511 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [107] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.172.527 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.172.583 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [45] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.172.608 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.172.629 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.172.733 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [94] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.172.752 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.172.766 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.172.776 [graph_manager.cc:2700][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [627] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.173.059 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [4] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:02.173.074 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:02.173.084 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:02.173.093 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [4] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:02.173.115 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:02.173.124 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CastRemovePass is [53] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:02.173.133 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:02.173.141 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [6] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:02.173.150 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [14] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:02.173.158 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [4] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:02.173.166 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [20] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:02.173.175 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [17] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:02.173.183 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [24] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:02.173.191 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [12] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:02.173.200 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [5] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:02.173.210 [graph_manager.cc:2741][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [415] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.173.219 [graph_manager.cc:2752][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.173.242 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.173.255 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.173.279 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.173.295 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.173.306 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.173.319 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.173.340 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.173.355 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.173.368 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.173.378 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.173.396 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.173.408 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.173.433 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.173.446 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.173.455 [graph_manager.cc:2810][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [218] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.173.505 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:38:02.173.517 [graph_manager.cc:2821][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [54] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.173.547 [graph_manager.cc:1087][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1407] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.174.258 [graph_manager.cc:1088][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [696] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.174.327 [graph_manager.cc:1089][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.174.349 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.174.367 [graph_manager.cc:1097][EVENT]167201 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:38:02.174.390 [graph_manager.cc:3325][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.184.718 [engine_place.cc:144][EVENT]167201 Run:The time cost of AIcoreEngine::CheckSupported is [10060] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.184.749 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.184.760 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.184.865 [graph_manager.cc:3351][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10459] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.184.884 [graph_manager.cc:3364][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.184.973 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [33] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.185.009 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.185.222 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [202] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.185.280 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [34] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.185.331 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.185.369 [graph_manager.cc:3405][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [471] micro second. [INFO] GE(164039,python):2024-01-10-11:38:02.185.388 [graph_manager.cc:3412][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. / - \ | / - \ | / - [INFO] GE(164039,python):2024-01-10-11:39:46.856.570 [graph_manager.cc:3422][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [104671165] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.856.643 [graph_manager.cc:3428][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.856.905 [graph_manager.cc:3467][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [234] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.856.927 [graph_manager.cc:3377][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [104672030] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.856.946 [graph_manager.cc:1106][EVENT]167201 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [104682564] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.856.959 [graph_manager.cc:1115][EVENT]167201 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:39:46.856.992 [graph_manager.cc:1130][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.027 [graph_manager.cc:1131][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.062 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.086 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.097 [graph_manager.cc:2837][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [52] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.299 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [43] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:39:46.857.313 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:39:46.857.323 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [8] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:39:46.857.333 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:39:46.857.342 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [12] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:39:46.857.351 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [22] micro second, call num is [12] [INFO] GE(164039,python):2024-01-10-11:39:46.857.380 [graph_manager.cc:2864][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [261] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.395 [graph_manager.cc:2872][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.420 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.436 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.454 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.470 [compile_nodes_pass.cc:88][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.480 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.490 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.621 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [121] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.681 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [47] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.737 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.759 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.777 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.787 [graph_manager.cc:2927][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [374] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.799 [graph_manager.cc:2937][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.816 [graph_manager.cc:2943][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.857.826 [graph_manager.cc:2950][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.858.073 [graph_manager.cc:2958][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [67] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.858.108 [graph_manager.cc:1132][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [1066] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.858.201 [graph_manager.cc:1135][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [79] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.858.252 [graph_manager.cc:2975][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [26] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.858.285 [graph_manager.cc:2981][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.858.300 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.858.309 [graph_manager.cc:2986][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.858.318 [graph_manager.cc:1136][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [94] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.858.705 [graph_manager.cc:3555][EVENT]167201 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [342] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.858.862 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [43] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.858.900 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.859.095 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [184] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.859.138 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [29] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.859.188 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.859.217 [graph_builder.cc:865][EVENT]167201 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [434] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:39:46.859.663 [logger.cc:1071] 167201 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:39:46.859.695 [task_generator.cc:804][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [90] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.859.794 [task_generator.cc:805][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [85] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.862.235 [task_generator.cc:814][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [2426] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.862.251 [task_generator.cc:954][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2647] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.862.330 [task_generator.cc:967][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [42] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:39:46.862.356 [logger.cc:1084] 167201 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:39:46.863.602 [graph_manager.cc:1152][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [5245] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.863.638 [graph_manager.cc:1164][EVENT]167201 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:39:46.863.680 [graph_manager.cc:1271][EVENT]167201 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [104717562] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.863.704 [graph_manager.cc:1272][EVENT]167201 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:39:46.864.029 [atrace_api.c:93](tid:167201) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:39:46.864.047 [atrace_api.c:95](tid:167201) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:39:46.894.047 [graph_converter.cc:838][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [11535] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.894.294 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [186] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.896.110 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [1790] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.896.616 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [477] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.896.642 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [504] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.896.936 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [281] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.897.036 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [77] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.897.119 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [65] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.897.707 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [550] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.897.975 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [239] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.897.997 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [263] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.898.077 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [69] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.898.150 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [59] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.898.221 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [60] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.898.493 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [259] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.898.733 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [223] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.898.751 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [242] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.898.828 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [67] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.898.897 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [57] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.898.917 [graph_converter.cc:849][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4818] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.899.811 [graph_converter.cc:853][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [882] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.902.146 [graph_converter.cc:857][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2313] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.902.656 [graph_converter.cc:862][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [454] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.990.122 [graph_var_manager.cc:1424][EVENT]167202 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:39:46.990.210 [graph_manager.cc:1248][EVENT]167202 PreRun:PreRun start: graph node size 4, session id 28, graph id 27, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:39:46.990.481 [atrace_api.c:28](tid:167202) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:39:46.990.507 [trace_rb_log.c:84](tid:167202) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:39:46.990.520 [atrace_api.c:32](tid:167202) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:39:46.990.538 [client_manager.cpp:157][SetProfilingCallback][tid:167202] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:39:46.990.962 [parallel_partitioner.cc:165][EVENT]167202 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.990.996 [parallel_partitioner.cc:178][EVENT]167202 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.991.044 [graph_prepare.cc:1378][EVENT]167202 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.991.222 [graph_manager.cc:1050][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [196] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.991.246 [graph_manager.cc:1052][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.991.383 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.991.414 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.991.462 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.991.475 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.991.521 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.991.534 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.991.552 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.991.644 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.991.664 [graph_manager.cc:1054][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [407] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.991.902 [graph_manager.cc:1055][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [225] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.992.913 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:39:46.992.938 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.992.949 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.992.959 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferShapePass is [323] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.992.968 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.992.977 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:39:46.992.985 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [15] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.992.994 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.993.002 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferValuePass is [11] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.994.788 [graph_manager.cc:1056][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2852] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.994.854 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.994.872 [graph_prepare.cc:1982][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [52] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.995.379 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:39:46.995.402 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.413 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.422 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferShapePass is [275] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.431 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.440 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [8] [INFO] GE(164039,python):2024-01-10-11:39:46.995.448 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.456 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.465 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.510 [graph_prepare.cc:1983][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [624] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.995.534 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.995.554 [graph_prepare.cc:1984][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [29] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.995.569 [graph_prepare.cc:1985][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.995.584 [graph_prepare.cc:1986][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.995.594 [graph_prepare.cc:1987][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.995.608 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.995.619 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.995.632 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.995.723 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.735 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.744 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.753 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.761 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.769 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.778 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.786 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.794 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.802 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.811 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.819 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SnapshotPass is [0] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.827 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.835 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.843 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.851 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:46.995.879 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.995.892 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.995.926 [graph_prepare.cc:1988][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [323] micro second. [INFO] GE(164039,python):2024-01-10-11:39:46.995.939 [graph_manager.cc:1065][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1120] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.008.192 [graph_manager.cc:1077][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12235] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.008.292 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.008.340 [graph_manager.cc:1080][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [112] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.016.312 [graph_manager.cc:1081][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [7955] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.016.365 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.016.379 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.016.391 [graph_manager.cc:1082][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.016.422 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.016.437 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.016.451 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.016.545 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [85] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.016.561 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.016.608 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.016.624 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.016.665 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [30] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.016.685 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.016.712 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.016.756 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.016.772 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.016.784 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.016.793 [graph_manager.cc:2700][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [376] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.016.922 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.016.935 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.016.945 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.016.954 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.016.963 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.016.971 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.016.980 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.016.988 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.016.996 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.017.004 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.017.013 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.017.021 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.017.029 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.017.038 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.017.046 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.017.056 [graph_manager.cc:2741][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [244] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.065 [graph_manager.cc:2752][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.087 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.098 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.121 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.136 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.149 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.160 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.178 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.191 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.204 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.215 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.227 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.240 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.260 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.273 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.283 [graph_manager.cc:2810][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [200] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.312 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.017.324 [graph_manager.cc:2821][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [32] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.352 [graph_manager.cc:1087][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [942] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.488 [graph_manager.cc:1088][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [123] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.534 [graph_manager.cc:1089][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [25] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.599 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.017.621 [graph_manager.cc:1097][EVENT]167202 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:39:47.017.647 [graph_manager.cc:3325][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.018.060 [engine_place.cc:144][EVENT]167202 Run:The time cost of AIcoreEngine::CheckSupported is [286] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.018.093 [engine_place.cc:144][EVENT]167202 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.018.103 [engine_place.cc:144][EVENT]167202 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.018.186 [graph_manager.cc:3351][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [525] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.018.205 [graph_manager.cc:3364][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.018.274 [engine_partitioner.cc:1139][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.018.293 [engine_partitioner.cc:1142][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.018.456 [engine_partitioner.cc:1148][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [153] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.018.499 [engine_partitioner.cc:1155][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [30] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.018.547 [engine_partitioner.cc:1164][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.018.580 [graph_manager.cc:3405][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [363] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.018.599 [graph_manager.cc:3412][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.232 [graph_manager.cc:3422][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [17618] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.271 [graph_manager.cc:3428][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.414 [graph_manager.cc:3467][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [121] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.432 [graph_manager.cc:3377][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [18215] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.447 [graph_manager.cc:1106][EVENT]167202 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [18808] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.459 [graph_manager.cc:1115][EVENT]167202 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:39:47.036.482 [graph_manager.cc:1130][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.516 [graph_manager.cc:1131][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.540 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.568 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.578 [graph_manager.cc:2837][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [47] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.666 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.036.678 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.036.688 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.036.697 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.036.706 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [7] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.036.714 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [9] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:39:47.036.724 [graph_manager.cc:2864][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [128] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.736 [graph_manager.cc:2872][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.755 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.769 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.784 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.797 [compile_nodes_pass.cc:88][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.807 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.817 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.896 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [70] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.923 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.936 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.949 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.962 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.971 [graph_manager.cc:2927][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [219] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.036.989 [graph_manager.cc:2937][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.037.003 [graph_manager.cc:2943][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.037.014 [graph_manager.cc:2950][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.037.204 [graph_manager.cc:2958][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.037.236 [graph_manager.cc:1132][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [706] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.037.306 [graph_manager.cc:1135][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [57] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.037.339 [graph_manager.cc:2975][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.037.372 [graph_manager.cc:2981][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.037.386 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.037.396 [graph_manager.cc:2986][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.037.405 [graph_manager.cc:1136][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [82] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.037.546 [graph_manager.cc:3555][EVENT]167202 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [109] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.037.638 [engine_partitioner.cc:1139][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.037.654 [engine_partitioner.cc:1142][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.037.814 [engine_partitioner.cc:1148][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [150] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.037.849 [engine_partitioner.cc:1155][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.037.888 [engine_partitioner.cc:1164][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [29] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.037.911 [graph_builder.cc:865][EVENT]167202 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [307] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:39:47.038.217 [logger.cc:1071] 167202 ModelBindStream: model_id=1856, stream_id=65, flag=0. [INFO] GE(164039,python):2024-01-10-11:39:47.038.246 [task_generator.cc:804][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [74] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.038.308 [task_generator.cc:805][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [49] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.038.981 [task_generator.cc:814][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [651] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.038.994 [task_generator.cc:954][EVENT]167202 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [823] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.039.054 [task_generator.cc:967][EVENT]167202 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [33] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:39:47.039.072 [logger.cc:1084] 167202 ModelUnbindStream: model_id=1856, stream_id=65, [INFO] GE(164039,python):2024-01-10-11:39:47.039.684 [graph_manager.cc:1152][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2254] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.039.712 [graph_manager.cc:1164][EVENT]167202 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:39:47.039.747 [graph_manager.cc:1271][EVENT]167202 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [48876] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.039.758 [graph_manager.cc:1272][EVENT]167202 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:39:47.040.074 [atrace_api.c:93](tid:167202) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:39:47.040.091 [atrace_api.c:95](tid:167202) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:39:47.051.698 [graph_converter.cc:838][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [3572] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.051.867 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [122] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.052.368 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [480] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.052.579 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [189] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.052.599 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [209] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.052.821 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [211] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.052.861 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.052.891 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.053.088 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [185] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.053.171 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [66] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.053.185 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [80] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.053.215 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.053.239 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.053.264 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.053.340 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [66] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.053.420 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [58] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.053.432 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [70] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.053.459 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.053.482 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.053.496 [graph_converter.cc:849][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1756] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.053.742 [graph_converter.cc:853][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [236] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.054.478 [graph_converter.cc:857][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [719] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.054.622 [graph_converter.cc:862][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [121] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.136.781 [graph_var_manager.cc:1424][EVENT]167201 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:39:47.136.870 [graph_manager.cc:1248][EVENT]167201 PreRun:PreRun start: graph node size 7, session id 29, graph id 28, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:39:47.137.120 [atrace_api.c:28](tid:167201) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:39:47.137.151 [trace_rb_log.c:84](tid:167201) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:39:47.137.165 [atrace_api.c:32](tid:167201) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:39:47.137.182 [client_manager.cpp:157][SetProfilingCallback][tid:167201] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:39:47.137.585 [parallel_partitioner.cc:165][EVENT]167201 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.137.624 [parallel_partitioner.cc:178][EVENT]167201 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.137.676 [graph_prepare.cc:1378][EVENT]167201 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.137.889 [graph_manager.cc:1050][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [234] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.137.915 [graph_manager.cc:1052][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.138.105 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.138.135 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.138.186 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.138.216 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.138.264 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.138.278 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.138.296 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.138.396 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.138.417 [graph_manager.cc:1054][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [489] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.138.652 [graph_manager.cc:1055][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [223] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.140.075 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [14] [INFO] GE(164039,python):2024-01-10-11:39:47.140.101 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.140.112 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [5] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.140.122 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [504] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.140.131 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [18] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.140.140 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [14] [INFO] GE(164039,python):2024-01-10-11:39:47.140.149 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.140.158 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [23] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.140.166 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.141.556 [graph_manager.cc:1056][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2885] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.141.630 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [6] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.141.648 [graph_prepare.cc:1982][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [61] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.142.359 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [14] [INFO] GE(164039,python):2024-01-10-11:39:47.142.385 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.395 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.405 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [438] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.423 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.432 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [4] micro second, call num is [14] [INFO] GE(164039,python):2024-01-10-11:39:47.142.441 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.449 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.458 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.487 [graph_prepare.cc:1983][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [826] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.142.511 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.142.523 [graph_prepare.cc:1984][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.142.537 [graph_prepare.cc:1985][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.142.552 [graph_prepare.cc:1986][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.142.563 [graph_prepare.cc:1987][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.142.578 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.142.589 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.142.603 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.142.721 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.733 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.741 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.750 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.758 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DropOutPass is [4] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.767 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.775 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.783 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.797 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.806 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.814 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.822 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.831 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.839 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.847 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.855 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.142.880 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.142.893 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.142.932 [graph_prepare.cc:1988][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [360] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.142.945 [graph_manager.cc:1065][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1360] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.155.740 [graph_manager.cc:1077][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12776] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.155.824 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.155.876 [graph_manager.cc:1080][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [99] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.159.715 [graph_manager.cc:1081][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3824] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.159.756 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.159.771 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.159.782 [graph_manager.cc:1082][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.159.815 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.159.831 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.159.846 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.159.892 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [24] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.159.906 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.159.923 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.159.936 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.159.984 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.003 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.033 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.068 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [24] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.083 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.095 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.105 [graph_manager.cc:2700][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [295] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.263 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.160.276 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.160.286 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.160.295 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.160.304 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.160.312 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CastRemovePass is [12] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.160.321 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.160.329 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.160.337 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.160.346 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.160.354 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.160.362 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [9] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.160.377 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.160.386 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.160.394 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.160.404 [graph_manager.cc:2741][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [281] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.413 [graph_manager.cc:2752][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.435 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.446 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.466 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.482 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.493 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.506 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.525 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.539 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.551 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.562 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.575 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.586 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.609 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.622 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.630 [graph_manager.cc:2810][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [200] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.669 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.160.681 [graph_manager.cc:2821][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [41] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.715 [graph_manager.cc:1087][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [913] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.847 [graph_manager.cc:1088][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [120] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.893 [graph_manager.cc:1089][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [27] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.911 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.160.953 [graph_manager.cc:1097][EVENT]167201 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:39:47.160.976 [graph_manager.cc:3325][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.161.474 [engine_place.cc:144][EVENT]167201 Run:The time cost of AIcoreEngine::CheckSupported is [384] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.161.498 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.161.508 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.161.602 [graph_manager.cc:3351][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [612] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.161.620 [graph_manager.cc:3364][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.161.717 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [50] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.161.737 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.161.989 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [242] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.162.040 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.162.093 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [41] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.162.129 [graph_manager.cc:3405][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [498] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.162.147 [graph_manager.cc:3412][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.172.836 [graph_manager.cc:3422][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [10675] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.172.871 [graph_manager.cc:3428][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.027 [graph_manager.cc:3467][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [135] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.054 [graph_manager.cc:3377][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [11423] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.070 [graph_manager.cc:1106][EVENT]167201 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [12102] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.083 [graph_manager.cc:1115][EVENT]167201 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:39:47.173.106 [graph_manager.cc:1130][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.139 [graph_manager.cc:1131][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.165 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.184 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.194 [graph_manager.cc:2837][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.294 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.173.306 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.173.315 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.173.324 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.173.333 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [8] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.173.342 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.173.351 [graph_manager.cc:2864][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [140] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.364 [graph_manager.cc:2872][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.383 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.398 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.413 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.427 [compile_nodes_pass.cc:88][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.438 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.454 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.566 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [102] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.594 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.607 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.621 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.633 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.642 [graph_manager.cc:2927][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [263] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.655 [graph_manager.cc:2937][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.669 [graph_manager.cc:2943][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.680 [graph_manager.cc:2950][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.902 [graph_manager.cc:2958][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [51] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.173.936 [graph_manager.cc:1132][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [783] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.174.005 [graph_manager.cc:1135][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [56] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.174.046 [graph_manager.cc:2975][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [24] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.174.078 [graph_manager.cc:2981][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.174.093 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.174.102 [graph_manager.cc:2986][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.174.111 [graph_manager.cc:1136][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [90] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.174.258 [graph_manager.cc:3555][EVENT]167201 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [108] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.174.370 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.174.387 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.174.586 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [182] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.174.624 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [26] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.174.667 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [32] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.174.692 [graph_builder.cc:865][EVENT]167201 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [369] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:39:47.175.022 [logger.cc:1071] 167201 ModelBindStream: model_id=64, stream_id=321, flag=0. [INFO] GE(164039,python):2024-01-10-11:39:47.175.053 [task_generator.cc:804][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [74] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.175.129 [task_generator.cc:805][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [62] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.175.813 [task_generator.cc:814][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [670] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.175.826 [task_generator.cc:954][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [848] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.175.887 [task_generator.cc:967][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [35] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:39:47.175.905 [logger.cc:1084] 167201 ModelUnbindStream: model_id=64, stream_id=321, [INFO] GE(164039,python):2024-01-10-11:39:47.176.081 [graph_manager.cc:1152][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [1939] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.176.099 [graph_manager.cc:1164][EVENT]167201 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:39:47.176.133 [graph_manager.cc:1271][EVENT]167201 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [38649] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.176.143 [graph_manager.cc:1272][EVENT]167201 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:39:47.176.460 [atrace_api.c:93](tid:167201) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:39:47.176.476 [atrace_api.c:95](tid:167201) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:39:47.183.742 [graph_converter.cc:838][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1693] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.183.826 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.184.448 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [605] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.184.724 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [251] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.184.745 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [273] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.184.799 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [43] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.184.842 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.184.877 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.185.083 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [195] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.185.186 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [86] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.185.200 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [101] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.185.238 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [27] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.185.267 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.185.298 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.185.404 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [96] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.185.494 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [79] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.185.506 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [91] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.185.541 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [26] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.185.570 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.185.584 [graph_converter.cc:849][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1803] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.185.899 [graph_converter.cc:853][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [306] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.186.900 [graph_converter.cc:857][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [981] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.187.105 [graph_converter.cc:862][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [180] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.276.439 [graph_var_manager.cc:1424][EVENT]167202 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:39:47.276.527 [graph_manager.cc:1248][EVENT]167202 PreRun:PreRun start: graph node size 7, session id 30, graph id 29, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:39:47.276.819 [atrace_api.c:28](tid:167202) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:39:47.276.845 [trace_rb_log.c:84](tid:167202) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:39:47.276.857 [atrace_api.c:32](tid:167202) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:39:47.276.875 [client_manager.cpp:157][SetProfilingCallback][tid:167202] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:39:47.277.297 [parallel_partitioner.cc:165][EVENT]167202 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.277.352 [parallel_partitioner.cc:178][EVENT]167202 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.277.400 [graph_prepare.cc:1378][EVENT]167202 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.277.593 [graph_manager.cc:1050][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [212] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.277.619 [graph_manager.cc:1052][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.277.836 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [4] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.277.870 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.277.923 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [40] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.277.937 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.277.983 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.277.996 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.278.012 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.278.106 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.278.128 [graph_manager.cc:1054][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [496] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.278.366 [graph_manager.cc:1055][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [225] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.279.764 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [14] [INFO] GE(164039,python):2024-01-10-11:39:47.279.792 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.279.804 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.279.814 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferShapePass is [498] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.279.824 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [18] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.279.832 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [14] [INFO] GE(164039,python):2024-01-10-11:39:47.279.841 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [15] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.279.849 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [23] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.279.865 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferValuePass is [7] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.281.256 [graph_manager.cc:1056][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2871] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.281.329 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondRemovePass is [6] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.281.348 [graph_prepare.cc:1982][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [61] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.282.104 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [14] [INFO] GE(164039,python):2024-01-10-11:39:47.282.129 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.141 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.150 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferShapePass is [457] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.159 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [15] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.168 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [14] [INFO] GE(164039,python):2024-01-10-11:39:47.282.176 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [10] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.185 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.193 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.251 [graph_prepare.cc:1983][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [889] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.282.275 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.282.287 [graph_prepare.cc:1984][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.282.301 [graph_prepare.cc:1985][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.282.315 [graph_prepare.cc:1986][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.282.327 [graph_prepare.cc:1987][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.282.342 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.282.354 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.282.368 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.282.484 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of EnterPass is [4] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.502 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.512 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.521 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.529 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.538 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.546 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.554 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.562 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of StopGradientPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.570 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.579 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.587 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.595 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.603 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.611 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.619 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.282.645 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.282.658 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.282.695 [graph_prepare.cc:1988][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [359] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.282.709 [graph_manager.cc:1065][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1422] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.295.460 [graph_manager.cc:1077][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12732] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.295.589 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.295.643 [graph_manager.cc:1080][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [149] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.299.390 [graph_manager.cc:1081][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3722] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.299.430 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.299.446 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.299.458 [graph_manager.cc:1082][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.299.489 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.299.505 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.299.520 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.299.551 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.299.564 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.299.581 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.299.594 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.299.642 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [38] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.299.661 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.299.691 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.299.724 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [24] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.299.740 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.299.752 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.299.761 [graph_manager.cc:2700][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [277] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.299.919 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.299.933 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AddNPass is [0] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.299.943 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.299.952 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.299.969 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.299.979 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CastRemovePass is [13] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.299.988 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.299.996 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [5] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.300.005 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.300.013 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [4] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.300.021 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.300.030 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.300.038 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.300.046 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.300.054 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.300.064 [graph_manager.cc:2741][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [285] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.073 [graph_manager.cc:2752][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.096 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.106 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.126 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.141 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.153 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.165 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.185 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.199 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.212 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.222 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.240 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.252 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.273 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.286 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.295 [graph_manager.cc:2810][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [203] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.331 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.300.343 [graph_manager.cc:2821][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.371 [graph_manager.cc:1087][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [894] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.504 [graph_manager.cc:1088][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [121] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.550 [graph_manager.cc:1089][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [26] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.567 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.300.611 [graph_manager.cc:1097][EVENT]167202 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:39:47.300.633 [graph_manager.cc:3325][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.301.100 [engine_place.cc:144][EVENT]167202 Run:The time cost of AIcoreEngine::CheckSupported is [354] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.301.124 [engine_place.cc:144][EVENT]167202 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.301.134 [engine_place.cc:144][EVENT]167202 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.301.226 [graph_manager.cc:3351][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [579] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.301.243 [graph_manager.cc:3364][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.301.313 [engine_partitioner.cc:1139][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.301.331 [engine_partitioner.cc:1142][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.301.575 [engine_partitioner.cc:1148][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [234] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.301.631 [engine_partitioner.cc:1155][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [35] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.301.683 [engine_partitioner.cc:1164][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [41] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.301.746 [graph_manager.cc:3405][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [492] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.301.765 [graph_manager.cc:3412][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.312.658 [graph_manager.cc:3422][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [10878] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.312.694 [graph_manager.cc:3428][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.312.847 [graph_manager.cc:3467][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [133] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.312.865 [graph_manager.cc:3377][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [11611] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.312.882 [graph_manager.cc:1106][EVENT]167202 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [12255] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.312.895 [graph_manager.cc:1115][EVENT]167202 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:39:47.312.918 [graph_manager.cc:1130][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.312.949 [graph_manager.cc:1131][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.312.974 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.312.992 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.002 [graph_manager.cc:2837][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.099 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.313.112 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.313.121 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.313.130 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.313.138 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.313.147 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [6] micro second, call num is [7] [INFO] GE(164039,python):2024-01-10-11:39:47.313.164 [graph_manager.cc:2864][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [145] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.177 [graph_manager.cc:2872][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.197 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.210 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.226 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.239 [compile_nodes_pass.cc:88][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.249 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.259 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.369 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [100] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.397 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.411 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.424 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.437 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.446 [graph_manager.cc:2927][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [253] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.458 [graph_manager.cc:2937][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.473 [graph_manager.cc:2943][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.483 [graph_manager.cc:2950][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.673 [graph_manager.cc:2958][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [49] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.735 [graph_manager.cc:1132][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [773] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.807 [graph_manager.cc:1135][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [56] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.847 [graph_manager.cc:2975][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.885 [graph_manager.cc:2981][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.899 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.909 [graph_manager.cc:2986][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.313.918 [graph_manager.cc:1136][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [94] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.314.068 [graph_manager.cc:3555][EVENT]167202 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [110] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.314.175 [engine_partitioner.cc:1139][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.314.192 [engine_partitioner.cc:1142][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.314.384 [engine_partitioner.cc:1148][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [181] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.314.422 [engine_partitioner.cc:1155][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [25] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.314.463 [engine_partitioner.cc:1164][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [30] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.314.487 [graph_builder.cc:865][EVENT]167202 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [355] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:39:47.314.817 [logger.cc:1071] 167202 ModelBindStream: model_id=1344, stream_id=1601, flag=0. [INFO] GE(164039,python):2024-01-10-11:39:47.314.847 [task_generator.cc:804][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [70] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.314.919 [task_generator.cc:805][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [60] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.315.639 [task_generator.cc:814][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [706] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.315.652 [task_generator.cc:954][EVENT]167202 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [876] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.315.711 [task_generator.cc:967][EVENT]167202 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [34] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:39:47.315.730 [logger.cc:1084] 167202 ModelUnbindStream: model_id=1344, stream_id=1601, [INFO] GE(164039,python):2024-01-10-11:39:47.315.905 [graph_manager.cc:1152][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [1956] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.315.924 [graph_manager.cc:1164][EVENT]167202 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:39:47.315.957 [graph_manager.cc:1271][EVENT]167202 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [38759] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.315.968 [graph_manager.cc:1272][EVENT]167202 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:39:47.316.285 [atrace_api.c:93](tid:167202) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:39:47.316.301 [atrace_api.c:95](tid:167202) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:39:47.323.504 [graph_converter.cc:838][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1687] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.323.586 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.324.212 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [609] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.324.487 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [252] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.324.507 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [272] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.324.561 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [42] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.324.594 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.324.627 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.324.830 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [192] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.324.931 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [84] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.324.944 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [98] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.324.981 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [27] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.325.010 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.325.041 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.325.147 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [96] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.325.235 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [78] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.325.247 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [90] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.325.282 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [26] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.325.310 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.325.324 [graph_converter.cc:849][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1782] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.325.622 [graph_converter.cc:853][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [288] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.326.657 [graph_converter.cc:857][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1017] micro second. [INFO] GE(164039,python):2024-01-10-11:39:47.326.873 [graph_converter.cc:862][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [181] micro second. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:39:47.331.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 1, total running count: 1 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:27.365.616 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:27.368.075 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 102 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:27.386.530 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 2, total running count: 2 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:27.779.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:27.781.779 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 103 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:27.795.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 3, total running count: 3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:27.795.613 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:27.797.505 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 104 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:27.810.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 4, total running count: 4 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:27.811.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:27.813.140 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 105 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:27.826.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 5, total running count: 5 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:27.826.789 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:27.828.616 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 106 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:27.841.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 6, total running count: 6 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:27.842.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:27.844.040 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 107 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:27.857.271 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 7, total running count: 7 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:27.857.602 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:27.860.412 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 108 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:27.872.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 8, total running count: 8 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:27.872.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:27.875.006 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 109 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:27.887.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 9, total running count: 9 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:27.888.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:27.890.663 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 110 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:27.903.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 10, total running count: 10 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:27.903.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:27.906.675 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 111 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:27.918.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 11, total running count: 11 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:27.918.678 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:27.921.011 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 112 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:27.933.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 12, total running count: 12 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:27.933.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:27.935.423 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 113 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:27.948.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 13, total running count: 13 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:27.948.666 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:27.950.833 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 114 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:27.963.343 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 14, total running count: 14 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:27.963.704 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:27.966.265 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 115 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:27.978.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 15, total running count: 15 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:27.978.649 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:27.980.539 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 116 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:27.993.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 16, total running count: 16 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:27.993.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:27.995.917 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 117 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.008.348 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 17, total running count: 17 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.008.690 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.010.340 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 118 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:28.023.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 18, total running count: 18 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.023.625 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.025.795 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 119 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:28.038.205 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 19, total running count: 19 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.038.533 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.041.199 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 120 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.053.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 20, total running count: 20 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:28.053.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.055.596 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 121 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.068.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 21, total running count: 21 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.068.443 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.070.947 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 122 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.083.275 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 22, total running count: 22 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.083.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.085.287 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 123 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.098.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 23, total running count: 23 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.098.347 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.100.728 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 124 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.113.018 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 24, total running count: 24 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.113.344 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.114.883 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 125 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.127.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 25, total running count: 25 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.128.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.130.340 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 126 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.142.789 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 26, total running count: 26 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.143.114 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.144.717 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 127 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.157.644 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 27, total running count: 27 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.157.988 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.160.089 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 128 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.172.594 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 28, total running count: 28 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.172.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.174.612 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 129 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.187.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 29, total running count: 29 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.187.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.190.106 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 130 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.202.349 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 30, total running count: 30 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.202.686 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.204.335 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 131 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.217.291 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 31, total running count: 31 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.217.611 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.219.731 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 132 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.232.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 32, total running count: 32 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.232.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.235.237 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 133 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.247.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 33, total running count: 33 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.247.680 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.249.605 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 134 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.262.336 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 34, total running count: 34 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:28.262.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.265.060 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 135 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.277.164 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 35, total running count: 35 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.277.508 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.279.312 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 136 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.292.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 36, total running count: 36 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.292.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.294.783 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 137 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:28.307.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 37, total running count: 37 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:28.307.440 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.309.270 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 138 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.321.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 38, total running count: 38 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.322.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.324.737 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 139 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.336.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 39, total running count: 39 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.337.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.338.996 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 140 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:28.351.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 40, total running count: 40 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.352.219 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.354.498 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 141 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.366.970 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 41, total running count: 41 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:28.367.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.369.900 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 142 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.381.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 42, total running count: 42 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.382.311 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.384.132 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 143 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.397.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 43, total running count: 43 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.397.404 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.399.467 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 144 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.412.018 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 44, total running count: 44 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:28.412.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.414.929 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 145 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.427.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 45, total running count: 45 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.427.387 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.429.304 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 146 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.441.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 46, total running count: 46 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.442.291 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.444.686 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 147 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.456.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 47, total running count: 47 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.457.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.458.811 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 148 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.472.065 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 48, total running count: 48 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.472.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.474.179 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 149 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.487.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 49, total running count: 49 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.487.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.489.640 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 150 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:28.502.134 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 50, total running count: 50 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.502.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.505.104 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 151 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.517.222 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 51, total running count: 51 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.517.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.519.298 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 152 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.532.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 52, total running count: 52 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.532.636 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.534.715 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 153 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.547.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 53, total running count: 53 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:28.547.748 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.550.134 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 154 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.562.381 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 54, total running count: 54 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.562.714 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.564.438 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 155 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.577.423 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 55, total running count: 55 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.577.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.579.757 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 156 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.592.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 56, total running count: 56 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.592.851 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.595.184 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 157 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.607.420 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 57, total running count: 57 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.607.752 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.609.554 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 158 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.622.385 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 58, total running count: 58 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.622.718 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.625.053 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 159 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.637.462 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 59, total running count: 59 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.637.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.640.433 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 160 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.652.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 60, total running count: 60 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.652.644 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.654.615 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 161 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.667.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 61, total running count: 61 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.667.596 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.670.059 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 162 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.682.286 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 62, total running count: 62 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.682.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.684.371 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 163 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.697.207 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 63, total running count: 63 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:28.697.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.699.788 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 164 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.712.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 64, total running count: 64 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.712.424 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.714.123 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 165 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.727.262 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 65, total running count: 65 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.727.608 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.729.473 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 166 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.742.116 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 66, total running count: 66 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.742.474 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.744.877 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 167 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.757.097 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 67, total running count: 67 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.757.437 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.759.039 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 168 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.772.053 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 68, total running count: 68 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.772.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.774.570 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 169 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.786.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 69, total running count: 69 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.787.247 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.788.865 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 170 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.801.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 70, total running count: 70 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.802.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.804.201 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 171 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:28.816.768 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 71, total running count: 71 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.817.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.819.673 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 172 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:28.831.543 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 72, total running count: 72 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.831.865 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.833.818 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 173 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.846.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 73, total running count: 73 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.846.841 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.849.294 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 174 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:28.861.484 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 74, total running count: 74 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.861.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.863.454 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 175 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.876.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 75, total running count: 75 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.876.762 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.878.859 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 176 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.891.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 76, total running count: 76 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.891.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.894.244 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 177 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.906.458 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 77, total running count: 77 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.906.795 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.908.591 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 178 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.921.458 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 78, total running count: 78 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:28.921.804 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.923.982 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 179 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.936.417 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 79, total running count: 79 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.936.736 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.938.297 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 180 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:28.951.338 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 80, total running count: 80 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.951.661 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.953.723 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 181 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.966.289 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 81, total running count: 81 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:28.966.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.969.051 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 182 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:28.981.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 82, total running count: 82 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:28.981.649 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.983.378 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 183 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:28.996.349 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 83, total running count: 83 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:28.996.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:28.998.854 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 184 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.011.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 84, total running count: 84 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.011.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.013.108 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 185 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.026.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 85, total running count: 85 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.026.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.028.462 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 186 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.041.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 86, total running count: 86 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.041.416 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.043.893 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 187 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.056.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 87, total running count: 87 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.056.413 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.058.122 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 188 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.071.002 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 88, total running count: 88 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.071.348 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.073.551 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 189 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.086.106 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 89, total running count: 89 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.086.447 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.089.005 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 190 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.100.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 90, total running count: 90 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.101.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.103.253 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 191 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.116.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 91, total running count: 91 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.116.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.118.626 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 192 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.131.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 92, total running count: 92 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.131.356 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.132.980 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 193 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.146.036 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 93, total running count: 93 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.146.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.148.401 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 194 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.160.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 94, total running count: 94 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.161.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.163.860 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 195 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.175.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 95, total running count: 95 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.176.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.178.057 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 196 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.190.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 96, total running count: 96 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.191.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.193.513 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 197 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.205.748 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 97, total running count: 97 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.206.072 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.207.793 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 198 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.220.624 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 98, total running count: 98 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.220.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.223.274 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 199 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.235.609 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 99, total running count: 99 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.235.928 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.237.622 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 200 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.250.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 100, total running count: 100 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.250.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.253.063 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 201 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.265.481 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 101, total running count: 101 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.265.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.268.516 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 202 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.280.567 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 102, total running count: 102 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.280.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.282.815 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 203 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.295.374 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 103, total running count: 103 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.295.711 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.298.326 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 204 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.310.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 104, total running count: 104 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.310.782 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.312.494 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 205 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.325.335 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 105, total running count: 105 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.325.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.327.914 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 206 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.340.495 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 106, total running count: 106 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.340.829 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.343.421 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 207 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.355.324 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 107, total running count: 107 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.355.650 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.357.784 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 208 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.370.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 108, total running count: 108 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.370.485 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.372.158 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 209 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.385.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 109, total running count: 109 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.385.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.387.725 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 210 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.400.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 110, total running count: 110 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.400.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.402.047 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 211 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.415.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 111, total running count: 111 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.415.443 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.417.400 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 212 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.430.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 112, total running count: 112 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.430.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.432.728 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 213 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.445.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 113, total running count: 113 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.445.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.446.954 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 214 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.460.092 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 114, total running count: 114 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.460.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.462.535 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 215 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.475.071 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 115, total running count: 115 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.475.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.478.048 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 216 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.490.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 116, total running count: 116 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.490.388 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.492.391 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 217 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.505.036 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 117, total running count: 117 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.505.372 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.507.835 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 218 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.520.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 118, total running count: 118 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.520.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.522.151 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 219 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.534.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 119, total running count: 119 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.535.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.537.532 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 220 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.549.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 120, total running count: 120 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.550.218 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.551.813 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 221 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.564.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 121, total running count: 121 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.565.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.567.221 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 222 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.579.738 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 122, total running count: 122 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.580.084 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.582.647 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 223 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.594.694 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 123, total running count: 123 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.595.036 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.596.815 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 224 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.609.556 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 124, total running count: 124 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.609.942 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.612.326 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 225 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.624.615 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 125, total running count: 125 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.624.937 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.626.674 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 226 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.639.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 126, total running count: 126 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.639.975 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.642.188 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 227 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.654.614 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 127, total running count: 127 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.654.943 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.656.542 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 228 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.669.531 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 128, total running count: 128 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.669.883 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.671.907 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 229 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.684.775 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 129, total running count: 129 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.685.102 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.687.308 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 230 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.699.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 130, total running count: 130 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.700.182 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.701.805 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 231 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.714.945 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 131, total running count: 131 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.715.269 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.717.319 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 232 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.729.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 132, total running count: 132 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.730.291 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.732.714 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 233 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.744.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 133, total running count: 133 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.745.222 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.746.885 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 234 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.759.856 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 134, total running count: 134 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.760.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.762.260 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 235 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.774.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 135, total running count: 135 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.775.232 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.777.756 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 236 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.789.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 136, total running count: 136 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.790.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.791.853 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 237 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.804.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 137, total running count: 137 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.805.246 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.807.199 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 238 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.819.989 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 138, total running count: 138 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.820.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.822.645 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 239 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:29.834.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 139, total running count: 139 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.835.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.836.912 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 240 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.849.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 140, total running count: 140 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.850.289 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.852.336 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 241 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.864.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 141, total running count: 141 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.865.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.867.720 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 242 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.884.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 142, total running count: 142 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.885.324 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.887.951 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 243 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.899.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 143, total running count: 143 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.900.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.902.326 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 244 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.914.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 144, total running count: 144 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.915.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.917.671 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 245 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.929.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 145, total running count: 145 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.930.241 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.931.939 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 246 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.944.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 146, total running count: 146 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.945.092 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.947.410 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 247 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.959.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 147, total running count: 147 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:29.960.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.961.750 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 248 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.974.650 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 148, total running count: 148 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:29.974.987 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.977.096 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 249 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:29.989.554 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 149, total running count: 149 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:29.989.885 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:29.992.477 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 250 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.004.576 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 150, total running count: 150 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:30.004.915 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.006.826 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 251 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.019.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 151, total running count: 151 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:30.020.052 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.022.260 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 252 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.034.733 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 152, total running count: 152 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:30.035.072 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.037.636 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 253 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.049.816 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 153, total running count: 153 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.050.145 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.051.866 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 254 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.064.663 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 154, total running count: 154 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:30.065.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.067.300 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 255 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.079.711 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 155, total running count: 155 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.080.061 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.081.595 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 256 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.094.700 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 156, total running count: 156 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.095.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.097.040 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 257 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.109.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 157, total running count: 157 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.110.016 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.112.548 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 258 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.124.629 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 158, total running count: 158 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.124.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.126.949 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 259 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.139.654 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 159, total running count: 159 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:30.140.009 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.142.382 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 260 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.154.654 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 160, total running count: 160 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.154.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.156.669 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 261 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.169.642 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 161, total running count: 161 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:30.170.015 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.172.027 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 262 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.184.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 162, total running count: 162 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.185.162 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.187.504 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 263 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:30.199.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 163, total running count: 163 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.200.164 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.201.814 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 264 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.214.756 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 164, total running count: 164 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.215.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.217.245 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 265 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.229.668 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 165, total running count: 165 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.230.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.232.657 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 266 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.244.775 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 166, total running count: 166 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.245.118 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.246.941 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 267 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.259.916 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 167, total running count: 167 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.260.239 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.262.369 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 268 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.274.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 168, total running count: 168 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.275.281 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.277.787 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 269 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.289.936 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 169, total running count: 169 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.290.281 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.292.058 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 270 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.304.887 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 170, total running count: 170 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.305.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.307.448 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 271 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.320.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 171, total running count: 171 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.320.336 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.322.803 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 272 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.335.027 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 172, total running count: 172 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.335.354 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.337.095 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 273 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.350.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 173, total running count: 173 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.350.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.352.533 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 274 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.364.936 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 174, total running count: 174 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.365.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.366.854 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 275 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.379.791 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 175, total running count: 175 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.380.127 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.382.320 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 276 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.394.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 176, total running count: 176 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.395.122 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.396.833 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 277 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.409.937 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 177, total running count: 177 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:30.410.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.412.158 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 278 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.424.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 178, total running count: 178 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.425.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.427.560 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 279 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.439.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 179, total running count: 179 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.440.249 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.441.799 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 280 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.454.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 180, total running count: 180 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.455.178 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.457.063 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 281 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:30.469.769 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 181, total running count: 181 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.470.108 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.472.463 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 282 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:30.484.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 182, total running count: 182 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.485.118 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.486.779 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 283 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.499.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 183, total running count: 183 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.500.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.502.232 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 284 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:30.514.789 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 184, total running count: 184 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.515.136 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.517.655 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 285 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.529.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 185, total running count: 185 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.530.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.531.933 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 286 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.544.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 186, total running count: 186 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.544.822 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.547.404 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 287 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.559.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 187, total running count: 187 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.559.789 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.561.790 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 288 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.574.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 188, total running count: 188 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.574.714 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.577.155 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 289 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.589.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 189, total running count: 189 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.589.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.591.373 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 290 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.604.158 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 190, total running count: 190 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:30.604.510 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.606.709 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 291 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.619.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 191, total running count: 191 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.619.395 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.621.041 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 292 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.633.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 192, total running count: 192 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.634.181 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.636.496 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 293 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.648.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 193, total running count: 193 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:30.649.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.650.858 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 294 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.663.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 194, total running count: 194 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.664.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.666.211 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 295 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.678.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 195, total running count: 195 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.679.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.681.537 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 296 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.693.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 196, total running count: 196 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.694.053 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.695.701 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 297 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.708.584 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 197, total running count: 197 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.708.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.711.228 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 298 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.723.642 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 198, total running count: 198 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:30.723.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.725.476 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 299 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.738.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 199, total running count: 199 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.738.905 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.740.940 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 300 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.753.530 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 200, total running count: 200 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.753.876 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.756.407 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 301 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.768.403 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 201, total running count: 201 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.768.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.770.675 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 302 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:30.783.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 202, total running count: 202 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.783.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.786.062 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 303 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.798.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 203, total running count: 203 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:30.798.556 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.800.235 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 304 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.813.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 204, total running count: 204 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.813.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.815.707 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 305 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.828.078 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 205, total running count: 205 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.828.420 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.829.981 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 306 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.843.058 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 206, total running count: 206 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.843.391 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.845.466 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 307 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.857.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 207, total running count: 207 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.858.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.859.741 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 308 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.872.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 208, total running count: 208 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:30.872.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.875.155 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 309 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.887.661 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 209, total running count: 209 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.887.987 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.890.613 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 310 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.902.734 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 210, total running count: 210 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:30.903.056 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.904.858 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 311 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.917.632 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 211, total running count: 211 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.917.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.920.154 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 312 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.932.522 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 212, total running count: 212 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:30.932.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.935.463 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 313 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.947.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 213, total running count: 213 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:30.947.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.949.767 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 314 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.962.389 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 214, total running count: 214 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:30.962.720 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.965.038 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 315 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.977.207 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 215, total running count: 215 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:30.977.552 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.979.300 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 316 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:30.992.250 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 216, total running count: 216 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:30.992.588 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:30.994.708 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 317 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.007.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 217, total running count: 217 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.007.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.010.150 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 318 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.022.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 218, total running count: 218 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.022.663 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.024.425 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 319 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.037.247 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 219, total running count: 219 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.037.587 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.039.833 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 320 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.052.036 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 220, total running count: 220 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.052.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.054.060 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 321 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.067.087 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 221, total running count: 221 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.067.425 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.069.285 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 322 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.081.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 222, total running count: 222 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.082.239 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.083.757 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 323 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.096.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 223, total running count: 223 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.097.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.099.192 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 324 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.111.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 224, total running count: 224 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.112.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.114.667 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 325 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.126.779 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 225, total running count: 225 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.127.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.128.949 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 326 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.141.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 226, total running count: 226 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.142.105 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.144.413 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 327 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.156.736 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 227, total running count: 227 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.157.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.158.782 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 328 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.171.740 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 228, total running count: 228 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.172.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.174.072 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 329 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.186.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 229, total running count: 229 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.187.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.189.362 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 330 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.201.737 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 230, total running count: 230 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.202.058 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.204.698 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 331 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.216.703 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 231, total running count: 231 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.217.020 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.218.841 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 332 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.231.563 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 232, total running count: 232 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.231.915 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.234.147 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 333 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.246.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 233, total running count: 233 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.246.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.248.494 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 334 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.261.570 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 234, total running count: 234 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.261.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.263.913 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 335 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.276.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 235, total running count: 235 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.276.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.279.351 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 336 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.291.717 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 236, total running count: 236 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.292.051 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.293.615 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 337 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.306.716 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 237, total running count: 237 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.307.036 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.309.067 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 338 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.321.728 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 238, total running count: 238 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.322.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.324.471 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 339 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.336.871 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 239, total running count: 239 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.337.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.339.834 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 340 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.351.967 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 240, total running count: 240 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.352.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.354.039 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 341 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.366.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 241, total running count: 241 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.367.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.369.621 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 342 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.382.128 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 242, total running count: 242 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.382.453 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.384.992 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 343 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.397.151 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 243, total running count: 243 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.397.488 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.399.290 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 344 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.412.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 244, total running count: 244 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.412.412 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.414.667 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 345 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.427.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 245, total running count: 245 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.427.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.430.100 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 346 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.442.044 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 246, total running count: 246 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.442.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.444.333 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 347 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.456.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 247, total running count: 247 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.457.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.459.772 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 348 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.471.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 248, total running count: 248 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.472.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.473.972 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 349 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.486.872 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 249, total running count: 249 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.487.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.489.439 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 350 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.501.887 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 250, total running count: 250 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.502.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.504.703 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 351 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.516.909 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 251, total running count: 251 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.517.238 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.518.926 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 352 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.532.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 252, total running count: 252 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.532.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.534.430 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 353 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.547.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 253, total running count: 253 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.547.382 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.549.792 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 354 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.562.009 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 254, total running count: 254 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.562.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.564.065 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 355 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.576.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 255, total running count: 255 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.577.295 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.579.607 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 356 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.592.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 256, total running count: 256 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.592.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.595.004 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 357 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.607.677 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 257, total running count: 257 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.608.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.610.339 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 358 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.622.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 258, total running count: 258 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.623.102 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.624.652 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 359 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.637.989 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 259, total running count: 259 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.638.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.639.994 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 360 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.652.993 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 260, total running count: 260 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.653.325 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.655.487 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 361 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.668.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 261, total running count: 261 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.668.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.670.987 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 362 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.683.181 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 262, total running count: 262 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.683.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.685.261 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 363 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.698.216 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 263, total running count: 263 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.698.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.700.673 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 364 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.713.252 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 264, total running count: 264 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.713.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.716.086 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 365 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.728.300 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 265, total running count: 265 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.728.638 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.730.428 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 366 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.743.233 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 266, total running count: 266 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.743.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.745.757 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 367 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.758.263 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 267, total running count: 267 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.758.587 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.761.012 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 368 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.773.246 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 268, total running count: 268 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.773.569 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.775.231 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 369 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.788.191 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 269, total running count: 269 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.788.516 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.790.699 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 370 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.803.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 270, total running count: 270 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.803.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.806.136 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 371 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.818.203 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 271, total running count: 271 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.818.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.820.348 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 372 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.833.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 272, total running count: 272 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.833.481 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.835.741 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 373 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.848.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 273, total running count: 273 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.848.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.849.970 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 374 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.863.135 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 274, total running count: 274 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.863.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.865.485 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 375 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.878.135 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 275, total running count: 275 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.878.476 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.880.980 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 376 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.893.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 276, total running count: 276 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.893.417 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.895.134 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 377 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.908.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 277, total running count: 277 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.908.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.910.603 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 378 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.923.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 278, total running count: 278 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.923.642 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.926.026 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 379 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.938.432 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 279, total running count: 279 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:31.938.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.940.382 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 380 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.953.394 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 280, total running count: 280 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:31.953.730 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.955.833 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 381 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.968.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 281, total running count: 281 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.968.757 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.971.219 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 382 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.983.313 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 282, total running count: 282 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:31.983.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:31.985.383 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 383 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:31.998.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 283, total running count: 283 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:31.998.623 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.000.680 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 384 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.013.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 284, total running count: 284 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.013.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.015.940 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 385 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.028.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 285, total running count: 285 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.028.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.030.249 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 386 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.043.084 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 286, total running count: 286 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.043.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.045.699 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 387 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.058.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 287, total running count: 287 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.058.354 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.060.578 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 388 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.072.905 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 288, total running count: 288 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.073.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.074.827 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 389 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.087.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 289, total running count: 289 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.088.200 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.090.280 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 390 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.102.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 290, total running count: 290 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.103.131 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.105.654 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 391 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.117.650 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 291, total running count: 291 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.118.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.119.906 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 392 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.132.531 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 292, total running count: 292 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.132.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.135.373 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 393 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.147.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 293, total running count: 293 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.147.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.149.726 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 394 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.162.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 294, total running count: 294 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.162.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.165.171 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 395 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.177.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 295, total running count: 295 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.177.618 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.179.394 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 396 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.192.236 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 296, total running count: 296 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.192.560 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.194.645 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 397 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.207.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 297, total running count: 297 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.207.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.210.069 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 398 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.222.219 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 298, total running count: 298 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.222.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.224.392 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 399 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.237.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 299, total running count: 299 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.237.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.239.853 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 400 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.252.231 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 300, total running count: 300 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.252.552 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.254.177 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 401 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.267.254 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 301, total running count: 301 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.267.569 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.269.530 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 402 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.282.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 302, total running count: 302 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.282.642 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.284.962 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 403 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.297.391 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 303, total running count: 303 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.297.739 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.299.258 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 404 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.312.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 304, total running count: 304 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.312.675 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.314.749 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 405 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.327.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 305, total running count: 305 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.327.580 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.330.098 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 406 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.342.099 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 306, total running count: 306 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.342.424 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.344.280 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 407 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.357.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 307, total running count: 307 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.357.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.359.561 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 408 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.372.031 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 308, total running count: 308 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.372.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.373.905 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 409 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.386.967 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 309, total running count: 309 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.387.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.389.355 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 410 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.402.087 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 310, total running count: 310 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.402.423 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.404.735 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 411 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.417.118 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 311, total running count: 311 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.417.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.420.013 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 412 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.431.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 312, total running count: 312 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.432.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.434.262 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 413 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.447.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 313, total running count: 313 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.447.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.449.684 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 414 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.461.925 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 314, total running count: 314 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.462.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.463.869 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 415 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.477.001 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 315, total running count: 315 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.477.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.479.232 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 416 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.492.008 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 316, total running count: 316 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.492.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.494.727 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 417 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.506.951 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 317, total running count: 317 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.507.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.509.031 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 418 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.521.812 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 318, total running count: 318 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.522.150 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.524.458 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 419 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.536.779 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 319, total running count: 319 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.537.096 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.538.645 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 420 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.551.637 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 320, total running count: 320 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.551.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.554.071 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 421 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.566.719 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 321, total running count: 321 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.567.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.569.605 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 422 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.581.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 322, total running count: 322 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.582.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.583.860 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 423 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.596.703 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 323, total running count: 323 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.597.032 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.599.293 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 424 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.611.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 324, total running count: 324 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.611.971 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.614.604 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 425 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.626.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 325, total running count: 325 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.626.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.628.846 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 426 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.641.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 326, total running count: 326 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.642.270 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.644.136 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 427 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.657.145 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 327, total running count: 327 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.657.541 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.659.775 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 428 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.672.642 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 328, total running count: 328 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.672.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.675.640 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 429 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.687.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 329, total running count: 329 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.688.302 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.690.294 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 430 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.703.134 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 330, total running count: 330 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.703.541 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.706.164 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 431 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.718.275 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 331, total running count: 331 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.718.606 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.720.428 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 432 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.733.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 332, total running count: 332 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.733.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.736.014 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 433 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.748.531 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 333, total running count: 333 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.748.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.751.099 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 434 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.763.711 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 334, total running count: 334 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.764.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.766.034 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 435 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.778.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 335, total running count: 335 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.779.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.780.889 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 436 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.793.832 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 336, total running count: 336 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.794.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.796.714 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 437 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.808.798 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 337, total running count: 337 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.809.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.811.373 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 438 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.824.292 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 338, total running count: 338 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.824.615 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.826.327 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 439 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.839.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 339, total running count: 339 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.839.806 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.842.271 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 440 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.854.644 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 340, total running count: 340 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.855.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.856.919 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 441 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.869.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 341, total running count: 341 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.870.046 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.872.676 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 442 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:32.884.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 342, total running count: 342 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.885.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.887.182 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 443 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.899.961 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 343, total running count: 343 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.900.302 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.903.005 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 444 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.914.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 344, total running count: 344 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.915.308 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.917.529 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 445 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:32.930.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 345, total running count: 345 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.930.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.932.112 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 446 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.945.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 346, total running count: 346 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.945.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.947.560 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 447 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.960.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 347, total running count: 347 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:32.960.590 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.962.375 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 448 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.975.168 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 348, total running count: 348 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:32.975.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.977.088 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 449 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.990.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 349, total running count: 349 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:32.990.482 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:32.992.715 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 450 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.005.096 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 350, total running count: 350 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.005.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.007.325 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 451 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.020.162 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 351, total running count: 351 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:33.020.511 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.022.996 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 452 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.035.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 352, total running count: 352 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.035.709 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.037.343 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 453 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.050.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 353, total running count: 353 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.050.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.052.745 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 454 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.065.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 354, total running count: 354 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.065.876 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.068.277 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 455 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.080.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 355, total running count: 355 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.080.649 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.082.700 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 456 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.095.187 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 356, total running count: 356 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.095.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.097.126 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 457 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.110.043 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 357, total running count: 357 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.110.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.112.594 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 458 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.125.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 358, total running count: 358 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:33.125.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.126.921 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 459 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.139.944 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 359, total running count: 359 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:33.140.287 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.142.572 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 460 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.154.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 360, total running count: 360 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:33.155.235 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.156.850 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 461 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.169.728 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 361, total running count: 361 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.170.145 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.172.101 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 462 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.184.854 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 362, total running count: 362 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.185.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.187.500 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 463 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:33.199.820 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 363, total running count: 363 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.200.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.201.977 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 464 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.214.816 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 364, total running count: 364 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:33.215.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.216.945 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 465 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:33.229.777 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 365, total running count: 365 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.230.122 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.232.613 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 466 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.244.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 366, total running count: 366 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.245.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.247.446 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 467 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:33.259.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 367, total running count: 367 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:33.260.335 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.261.888 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 468 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.275.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 368, total running count: 368 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.275.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.277.501 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 469 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.289.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 369, total running count: 369 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.290.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.292.038 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 470 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.304.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 370, total running count: 370 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.305.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.307.476 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 471 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.319.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 371, total running count: 371 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:33.320.286 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.321.884 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 472 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.334.992 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 372, total running count: 372 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.335.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.337.365 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 473 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.349.928 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 373, total running count: 373 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.350.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.352.992 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 474 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:33.364.971 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 374, total running count: 374 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.365.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.367.371 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 475 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:33.379.882 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 375, total running count: 375 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.380.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.381.823 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 476 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:33.394.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 376, total running count: 376 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:33.395.259 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.397.387 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 477 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.409.806 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 377, total running count: 377 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:33.410.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.412.887 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 478 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.424.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 378, total running count: 378 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:33.425.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.427.239 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 479 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.439.844 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 379, total running count: 379 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.440.182 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.442.767 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 480 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.454.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 380, total running count: 380 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.455.249 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.456.952 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 481 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.469.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 381, total running count: 381 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.470.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.472.449 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 482 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.484.679 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 382, total running count: 382 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.485.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.486.917 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 483 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.499.742 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 383, total running count: 383 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.500.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.502.630 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 484 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:33.514.798 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 384, total running count: 384 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.515.149 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.516.973 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 485 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.529.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 385, total running count: 385 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.530.319 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.532.586 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 486 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:33.544.926 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 386, total running count: 386 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.545.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.546.943 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 487 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.559.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 387, total running count: 387 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.560.301 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.562.398 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 488 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.574.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 388, total running count: 388 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.575.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.577.824 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 489 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.589.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 389, total running count: 389 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:33.590.222 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.592.057 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 490 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:33.604.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 390, total running count: 390 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.605.122 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.607.456 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 491 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.619.733 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 391, total running count: 391 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.620.048 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.621.769 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 492 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:33.634.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 392, total running count: 392 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.634.903 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.637.437 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 493 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.649.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 393, total running count: 393 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.649.817 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.651.866 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 494 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.664.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 394, total running count: 394 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.664.612 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.667.272 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 495 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.679.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 395, total running count: 395 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.679.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.681.672 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 496 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.694.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 396, total running count: 396 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:33.694.854 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.697.252 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 497 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:33.709.236 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 397, total running count: 397 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.709.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.711.522 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 498 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.724.165 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 398, total running count: 398 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:33.724.495 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.727.000 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 499 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.739.033 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 399, total running count: 399 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.739.398 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.741.408 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 500 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.753.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 400, total running count: 400 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:33.754.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.756.864 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 501 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.768.704 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 401, total running count: 401 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.769.154 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.771.172 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 502 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:33.783.757 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 402, total running count: 402 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.784.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.786.729 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 503 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.798.624 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 403, total running count: 403 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:33.798.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.801.143 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 504 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:33.813.515 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 404, total running count: 404 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.813.882 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.816.528 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 505 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.828.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 405, total running count: 405 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:33.828.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.830.761 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 506 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.843.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 406, total running count: 406 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.843.877 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.846.207 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 507 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.858.218 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 407, total running count: 407 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.858.629 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.860.631 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 508 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:33.873.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 408, total running count: 408 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.873.681 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.876.169 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 509 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.888.191 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 409, total running count: 409 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.888.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.890.454 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 510 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.902.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 410, total running count: 410 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.903.416 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.905.908 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 511 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:33.918.027 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 411, total running count: 411 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.918.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.920.174 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 512 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.932.888 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 412, total running count: 412 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.933.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.935.704 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 513 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:33.947.677 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 413, total running count: 413 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.948.030 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.949.855 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 514 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.962.481 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 414, total running count: 414 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.962.935 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.965.408 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 515 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.977.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 415, total running count: 415 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.977.887 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.979.828 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 516 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:33.992.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 416, total running count: 416 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:33.992.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:33.995.246 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 517 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.007.486 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 417, total running count: 417 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.007.915 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.009.586 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 518 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.022.482 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 418, total running count: 418 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.022.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.025.162 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 519 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.037.644 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 419, total running count: 419 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.038.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.039.557 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 520 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.052.514 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 420, total running count: 420 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.052.859 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.055.244 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 521 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.067.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 421, total running count: 421 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.067.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.069.653 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 522 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.082.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 422, total running count: 422 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.082.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.085.126 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 523 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.097.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 423, total running count: 423 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.097.481 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.099.344 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 524 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.111.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 424, total running count: 424 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.112.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.114.692 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 525 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.126.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 425, total running count: 425 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:34.127.318 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.129.088 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 526 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:34.141.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 426, total running count: 426 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:34.142.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.144.635 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 527 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.156.971 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 427, total running count: 427 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.157.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.158.907 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 528 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.171.850 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 428, total running count: 428 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.172.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.174.326 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 529 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.186.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 429, total running count: 429 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.187.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.189.985 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 530 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.201.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 430, total running count: 430 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:34.202.019 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.204.501 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 531 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:34.216.728 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 431, total running count: 431 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.217.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.218.908 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 532 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.231.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 432, total running count: 432 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.232.000 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.234.324 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 533 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.246.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 433, total running count: 433 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.247.000 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.248.616 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 534 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.261.523 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 434, total running count: 434 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.261.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.263.897 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 535 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.276.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 435, total running count: 435 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.276.666 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.278.263 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 536 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.291.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 436, total running count: 436 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.291.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.293.703 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 537 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.306.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 437, total running count: 437 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.306.527 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.308.069 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 538 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:34.321.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 438, total running count: 438 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.321.419 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.323.427 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 539 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.335.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 439, total running count: 439 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.336.278 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.338.864 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 540 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.350.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 440, total running count: 440 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.351.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.353.213 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 541 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.366.124 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 441, total running count: 441 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.366.474 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.368.934 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 542 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.381.164 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 442, total running count: 442 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:34.381.515 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.383.383 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 543 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.396.140 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 443, total running count: 443 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.396.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.398.929 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 544 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.411.008 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 444, total running count: 444 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:34.411.338 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.413.178 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 545 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:34.426.000 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 445, total running count: 445 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.426.330 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.428.616 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 546 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:34.441.056 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 446, total running count: 446 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.441.395 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.442.924 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 547 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.456.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 447, total running count: 447 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.456.389 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.458.279 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 548 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.470.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 448, total running count: 448 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.471.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.473.703 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 549 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:34.485.804 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 449, total running count: 449 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.486.149 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.488.318 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 550 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.500.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 450, total running count: 450 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.501.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.502.798 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 551 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.515.572 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 451, total running count: 451 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.515.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.518.377 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 552 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.530.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 452, total running count: 452 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:34.530.837 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.532.869 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 553 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.545.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 453, total running count: 453 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.546.046 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.548.543 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 554 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:34.560.531 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 454, total running count: 454 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.560.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.562.997 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 555 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.575.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 455, total running count: 455 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:34.575.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.577.469 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 556 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.590.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 456, total running count: 456 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.590.690 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.592.939 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 557 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:34.605.235 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 457, total running count: 457 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.605.637 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.607.333 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 558 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:34.620.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 458, total running count: 458 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.620.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.622.981 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 559 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.635.278 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 459, total running count: 459 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.635.754 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.638.317 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 560 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:34.650.330 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 460, total running count: 460 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.650.666 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.652.676 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 561 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:34.665.226 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 461, total running count: 461 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.665.569 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.668.100 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 562 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.680.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 462, total running count: 462 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.680.556 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.682.240 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 563 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.695.105 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 463, total running count: 463 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.695.447 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.697.566 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 564 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.710.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 464, total running count: 464 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.710.447 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.712.073 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 565 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.725.171 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 465, total running count: 465 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.725.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.727.565 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 566 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.740.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 466, total running count: 466 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.740.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.743.021 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 567 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.754.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 467, total running count: 467 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.755.247 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.757.309 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 568 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.769.887 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 468, total running count: 468 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:40:34.770.256 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_1 execution count: 1, execution time: 182521 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:40:34.770.440 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_1 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:40:34.770.613 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty epoch: 1 step: 468, loss is 2.3165836334228516 [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:40:34.771.762 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.h:76] ContinueSend] continue send at the beginning of the epoch [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:40:34.773.073 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:40:34.773.140 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:40:34.773.182 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_1 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.773.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.775.219 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 569 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.788.331 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 1, total running count: 469 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.788.649 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.790.773 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 570 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.803.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 2, total running count: 470 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.803.622 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.805.204 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 571 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.818.343 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 3, total running count: 471 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:34.818.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.820.678 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 572 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.833.277 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 4, total running count: 472 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.833.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.836.113 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 573 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.848.171 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 5, total running count: 473 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.848.485 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.850.412 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 574 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.863.150 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 6, total running count: 474 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:34.863.479 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.865.911 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 575 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.878.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 7, total running count: 475 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.878.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.880.251 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 576 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.893.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 8, total running count: 476 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.893.630 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.895.802 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 577 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.908.135 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 9, total running count: 477 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.908.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.910.155 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 578 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.923.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 10, total running count: 478 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.923.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.926.252 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 579 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.938.024 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 11, total running count: 479 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.938.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.940.592 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 580 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.952.925 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 12, total running count: 480 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:34.953.251 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.954.845 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 581 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:34.967.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 13, total running count: 481 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.968.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.969.893 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 582 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.983.051 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 14, total running count: 482 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.983.374 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:34.985.459 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 583 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:34.998.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 15, total running count: 483 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:34.998.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.000.995 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 584 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.012.878 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 16, total running count: 484 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:35.013.212 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.015.256 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 585 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.027.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 17, total running count: 485 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.028.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.030.666 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 586 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.042.900 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 18, total running count: 486 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:35.043.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.045.087 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 587 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.057.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 19, total running count: 487 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:35.058.192 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.060.519 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 588 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.072.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 20, total running count: 488 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.073.115 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.074.759 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 589 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.087.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 21, total running count: 489 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.087.865 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.090.218 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 590 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.102.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 22, total running count: 490 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:35.102.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.104.518 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 591 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.117.241 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 23, total running count: 491 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:35.117.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.119.884 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 592 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:35.131.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 24, total running count: 492 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.132.289 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.134.316 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 593 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.146.792 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 25, total running count: 493 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.147.115 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.149.772 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 594 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.161.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 26, total running count: 494 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.163.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.165.286 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 595 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.177.839 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 27, total running count: 495 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:35.178.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.180.610 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 596 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.193.636 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 28, total running count: 496 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.194.008 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.196.059 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 597 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.208.668 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 29, total running count: 497 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.209.002 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.211.487 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 598 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:35.223.666 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 30, total running count: 498 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.224.002 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.225.765 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 599 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.238.657 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 31, total running count: 499 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:35.239.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.241.090 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 600 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.253.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 32, total running count: 500 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.253.820 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.255.423 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 601 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.268.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 33, total running count: 501 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.268.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.270.883 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 602 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.282.987 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 34, total running count: 502 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:35.283.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.285.275 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 603 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.298.210 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 35, total running count: 503 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.298.612 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.300.668 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 604 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.313.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 36, total running count: 504 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:35.313.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.316.025 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 605 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.328.244 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 37, total running count: 505 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:35.328.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.330.358 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 606 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.343.177 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 38, total running count: 506 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.343.515 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.345.727 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 607 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.358.043 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 39, total running count: 507 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.358.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.360.024 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 608 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.372.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 40, total running count: 508 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.373.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.375.576 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 609 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.387.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 41, total running count: 509 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.388.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.389.917 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 610 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:35.402.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 42, total running count: 510 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.402.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.405.328 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 611 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.417.935 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 43, total running count: 511 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:35.418.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.420.747 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 612 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.432.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 44, total running count: 512 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:35.433.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.434.993 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 613 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.447.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 45, total running count: 513 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.448.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.450.401 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 614 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:35.462.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 46, total running count: 514 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:35.463.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.465.798 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 615 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.478.382 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 47, total running count: 515 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.478.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.481.331 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 616 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.493.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 48, total running count: 516 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:35.493.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.495.651 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 617 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.508.392 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 49, total running count: 517 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:35.508.728 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.511.087 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 618 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.523.216 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 50, total running count: 518 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:35.524.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.526.578 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 619 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.538.498 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 51, total running count: 519 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:35.539.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.542.034 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 620 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.554.563 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 52, total running count: 520 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:35.555.164 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.557.425 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 621 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.569.925 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 53, total running count: 521 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.570.395 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.572.773 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 622 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.584.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 54, total running count: 522 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.585.525 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.588.254 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 623 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.600.046 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 55, total running count: 523 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.600.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.602.530 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 624 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.615.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 56, total running count: 524 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.615.791 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.617.968 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 625 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.630.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 57, total running count: 525 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:35.630.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.633.271 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 626 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:35.645.412 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 58, total running count: 526 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.645.903 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.647.549 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 627 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.660.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 59, total running count: 527 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.660.933 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.662.982 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 628 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.675.439 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 60, total running count: 528 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.675.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.678.543 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 629 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:35.690.510 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 61, total running count: 529 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.690.882 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.692.867 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 630 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.705.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 62, total running count: 530 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:35.705.674 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.708.154 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 631 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.720.112 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 63, total running count: 531 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.720.739 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.722.366 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 632 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.735.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 64, total running count: 532 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.735.833 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.737.752 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 633 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.750.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 65, total running count: 533 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.750.887 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.753.068 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 634 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.765.486 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 66, total running count: 534 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:35.765.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.768.500 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 635 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.780.399 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 67, total running count: 535 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:35.784.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.786.203 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 636 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.798.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 68, total running count: 536 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.801.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.802.742 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 637 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.815.531 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 69, total running count: 537 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.816.567 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.818.229 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 638 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.831.314 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 70, total running count: 538 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.833.331 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.834.951 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 639 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.848.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 71, total running count: 539 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:35.849.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.851.494 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 640 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.863.832 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 72, total running count: 540 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.864.510 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.866.887 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 641 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.879.027 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 73, total running count: 541 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.879.809 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.882.277 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 642 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.894.381 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 74, total running count: 542 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.894.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.896.643 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 643 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:35.909.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 75, total running count: 543 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.910.145 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.911.999 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 644 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:35.924.724 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 76, total running count: 544 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.925.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.927.435 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 645 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.940.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 77, total running count: 545 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:35.940.459 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.942.875 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 646 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.955.138 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 78, total running count: 546 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.959.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.961.658 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 647 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:35.974.335 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 79, total running count: 547 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:35.974.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.977.168 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 648 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.989.660 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 80, total running count: 548 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:35.990.181 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:35.992.553 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 649 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.004.713 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 81, total running count: 549 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.005.217 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.006.734 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 650 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.019.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 82, total running count: 550 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.020.202 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.022.236 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 651 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.035.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 83, total running count: 551 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.035.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.037.744 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 652 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.050.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 84, total running count: 552 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.050.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.052.076 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 653 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.065.015 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 85, total running count: 553 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.065.396 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.067.514 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 654 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.079.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 86, total running count: 554 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.080.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.082.965 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 655 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.095.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 87, total running count: 555 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.097.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.099.483 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 656 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.111.925 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 88, total running count: 556 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.112.572 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.114.914 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 657 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.127.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 89, total running count: 557 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:36.127.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.130.352 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 658 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.142.423 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 90, total running count: 558 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.142.796 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.144.549 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 659 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.157.300 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 91, total running count: 559 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.157.794 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.159.939 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 660 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.172.533 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 92, total running count: 560 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.172.883 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.175.400 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 661 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.187.574 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 93, total running count: 561 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.187.949 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.189.644 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 662 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.202.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 94, total running count: 562 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.203.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.205.075 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 663 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.217.890 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 95, total running count: 563 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.218.243 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.220.648 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 664 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.232.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 96, total running count: 564 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.233.153 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.234.943 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 665 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.247.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 97, total running count: 565 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.248.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.250.472 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 666 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.262.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 98, total running count: 566 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.263.421 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.265.885 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 667 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:36.277.872 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 99, total running count: 567 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.278.515 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.281.288 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 668 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.293.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 100, total running count: 568 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.293.495 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.295.549 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 669 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.308.260 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 101, total running count: 569 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.308.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.310.945 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 670 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.323.077 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 102, total running count: 570 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.323.466 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.325.296 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 671 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.337.975 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 103, total running count: 571 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.338.532 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.340.689 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 672 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:36.353.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 104, total running count: 572 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.353.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.355.949 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 673 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.368.239 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 105, total running count: 573 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.368.695 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.371.317 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 674 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.383.394 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 106, total running count: 574 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.383.738 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.385.649 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 675 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.399.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 107, total running count: 575 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.399.714 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.402.223 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 676 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.415.633 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 108, total running count: 576 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.415.985 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.417.787 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 677 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.431.719 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 109, total running count: 577 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.432.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.434.227 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 678 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.447.455 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 110, total running count: 578 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.447.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.449.475 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 679 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.463.347 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 111, total running count: 579 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.463.681 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.466.084 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 680 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.478.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 112, total running count: 580 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.479.271 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.481.581 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 681 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.494.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 113, total running count: 581 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.494.790 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.496.924 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 682 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.515.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 114, total running count: 582 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:36.515.586 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.518.304 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 683 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.531.799 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 115, total running count: 583 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.532.139 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.533.685 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 684 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.547.514 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 116, total running count: 584 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.547.861 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.550.259 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 685 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.563.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 117, total running count: 585 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.563.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.565.641 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 686 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.579.036 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 118, total running count: 586 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.579.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.581.047 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 687 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.594.474 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 119, total running count: 587 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.594.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.596.593 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 688 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.610.212 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 120, total running count: 588 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.610.554 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.613.245 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 689 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.626.149 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 121, total running count: 589 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.626.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.628.500 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 690 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.641.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 122, total running count: 590 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:36.642.236 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.643.924 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 691 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.657.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 123, total running count: 591 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.658.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.660.542 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 692 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.673.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 124, total running count: 592 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.673.721 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.676.020 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 693 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.688.844 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 125, total running count: 593 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.689.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.691.485 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 694 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:36.704.541 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 126, total running count: 594 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.704.943 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.706.948 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 695 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.720.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 127, total running count: 595 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.720.871 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.723.534 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 696 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:36.737.171 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 128, total running count: 596 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.737.547 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.740.265 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 697 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.752.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 129, total running count: 597 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.753.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.754.574 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 698 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.768.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 130, total running count: 598 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.768.858 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.771.085 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 699 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.784.210 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 131, total running count: 599 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.784.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.786.512 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 700 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.799.961 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 132, total running count: 600 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.800.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.802.032 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 701 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.815.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 133, total running count: 601 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.816.036 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.818.693 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 702 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.831.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 134, total running count: 602 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.832.001 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.834.215 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 703 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.849.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 135, total running count: 603 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.850.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.851.828 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 704 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.865.563 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 136, total running count: 604 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.865.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.868.348 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 705 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.881.817 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 137, total running count: 605 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.882.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.883.770 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 706 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.897.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 138, total running count: 606 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.897.829 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.900.386 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 707 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.913.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 139, total running count: 607 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.913.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.915.808 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 708 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.929.079 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 140, total running count: 608 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.929.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.931.249 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 709 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.949.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 141, total running count: 609 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:36.950.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.952.455 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 710 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.965.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 142, total running count: 610 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:36.965.596 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.967.815 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 711 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.981.053 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 143, total running count: 611 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:36.981.404 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.983.178 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 712 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:36.996.737 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 144, total running count: 612 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:36.997.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:36.999.725 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 713 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.012.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 145, total running count: 613 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.012.744 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.015.119 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 714 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.028.120 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 146, total running count: 614 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:37.028.455 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.030.412 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 715 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.043.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 147, total running count: 615 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.044.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.045.799 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 716 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.059.489 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 148, total running count: 616 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.059.812 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.061.375 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 717 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.075.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 149, total running count: 617 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.075.738 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.077.983 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 718 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.091.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 150, total running count: 618 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:37.091.623 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.093.261 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 719 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.107.356 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 151, total running count: 619 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:37.107.759 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.110.058 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 720 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.123.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 152, total running count: 620 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.123.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.125.397 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 721 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:37.138.832 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 153, total running count: 621 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.139.216 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.140.772 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 722 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.154.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 154, total running count: 622 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.155.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.157.354 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 723 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.170.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 155, total running count: 623 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:37.170.837 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.172.614 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 724 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.185.818 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 156, total running count: 624 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.186.147 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.187.881 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 725 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.201.304 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 157, total running count: 625 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:37.201.637 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.203.316 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 726 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.217.131 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 158, total running count: 626 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.217.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.219.972 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 727 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.232.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 159, total running count: 627 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.233.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.235.456 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 728 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:37.248.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 160, total running count: 628 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:37.248.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.250.833 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 729 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.264.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 161, total running count: 629 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.264.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.266.199 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 730 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.279.597 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 162, total running count: 630 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.279.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.281.571 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 731 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.295.319 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 163, total running count: 631 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.295.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.297.970 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 732 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.310.889 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 164, total running count: 632 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.311.239 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.313.516 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 733 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:37.326.653 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 165, total running count: 633 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.327.019 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.328.861 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 734 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.342.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 166, total running count: 634 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.342.704 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.344.324 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 735 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.357.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 167, total running count: 635 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.358.251 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.360.813 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 736 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.373.563 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 168, total running count: 636 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.373.955 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.376.270 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 737 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.389.178 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 169, total running count: 637 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.389.526 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.391.559 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 738 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.405.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 170, total running count: 638 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.405.407 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.406.950 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 739 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.420.860 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 171, total running count: 639 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.421.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.423.508 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 740 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.436.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 172, total running count: 640 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.436.879 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.438.821 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 741 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.452.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 173, total running count: 641 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:37.452.742 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.454.281 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 742 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.468.224 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 174, total running count: 642 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.468.584 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.470.983 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 743 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:37.483.968 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 175, total running count: 643 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.484.348 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.486.358 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 744 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:37.499.447 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 176, total running count: 644 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.499.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.501.679 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 745 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.515.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 177, total running count: 645 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.515.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.517.059 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 746 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.530.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 178, total running count: 646 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.530.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.532.464 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 747 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.546.131 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 179, total running count: 647 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.546.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.549.004 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 748 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.561.657 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 180, total running count: 648 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.561.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.564.418 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 749 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.577.059 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 181, total running count: 649 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.577.387 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.579.929 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 750 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.592.679 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 182, total running count: 650 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.593.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.595.305 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 751 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.608.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 183, total running count: 651 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.608.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.610.656 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 752 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.624.018 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 184, total running count: 652 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.624.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.626.148 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 753 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.639.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 185, total running count: 653 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.640.141 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.642.575 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 754 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.655.614 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 186, total running count: 654 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.655.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.657.885 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 755 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.671.560 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 187, total running count: 655 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.671.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.674.387 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 756 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.686.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 188, total running count: 656 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.687.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.689.798 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 757 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.702.724 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 189, total running count: 657 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.703.066 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.705.096 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 758 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.718.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 190, total running count: 658 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.718.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.720.480 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 759 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.733.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 191, total running count: 659 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.734.061 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.735.890 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 760 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.749.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 192, total running count: 660 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.749.708 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.751.335 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 761 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.764.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 193, total running count: 661 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.765.238 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.767.814 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 762 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.780.629 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 194, total running count: 662 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.780.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.783.287 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 763 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.796.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 195, total running count: 663 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.796.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.798.810 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 764 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.811.915 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 196, total running count: 664 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.812.270 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.814.184 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 765 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.827.713 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 197, total running count: 665 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.828.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.829.659 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 766 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.843.246 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 198, total running count: 666 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.843.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.845.098 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 767 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.858.915 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 199, total running count: 667 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.859.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.861.712 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 768 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.874.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 200, total running count: 668 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.875.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.877.278 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 769 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.890.521 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 201, total running count: 669 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.890.859 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.892.647 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 770 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.906.247 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 202, total running count: 670 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:37.906.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.909.230 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 771 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.921.933 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 203, total running count: 671 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.922.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.924.624 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 772 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.937.488 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 204, total running count: 672 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.937.860 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.940.014 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 773 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.953.059 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 205, total running count: 673 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.953.440 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.955.475 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 774 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.968.708 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 206, total running count: 674 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.969.042 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.970.873 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 775 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:37.984.379 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 207, total running count: 675 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:37.984.724 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:37.987.402 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 776 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:37.999.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 208, total running count: 676 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.000.210 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.002.825 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 777 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.015.552 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 209, total running count: 677 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.015.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.018.321 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 778 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.031.213 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 210, total running count: 678 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.031.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.033.640 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 779 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.047.013 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 211, total running count: 679 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.047.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.048.906 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 780 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.062.413 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 212, total running count: 680 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.062.777 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.064.392 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 781 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.077.968 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 213, total running count: 681 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.078.354 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.081.021 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 782 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.093.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 214, total running count: 682 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.093.970 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.096.538 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 783 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.109.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 215, total running count: 683 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.109.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.111.950 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 784 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.125.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 216, total running count: 684 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.125.395 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.127.385 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 785 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.140.732 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 217, total running count: 685 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.141.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.142.756 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 786 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.156.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 218, total running count: 686 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.156.526 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.158.197 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 787 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.171.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 219, total running count: 687 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.172.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.174.708 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 788 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.187.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 220, total running count: 688 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.187.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.190.162 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 789 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.203.048 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 221, total running count: 689 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.203.403 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.205.533 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 790 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.218.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 222, total running count: 690 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.219.028 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.220.907 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 791 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.234.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 223, total running count: 691 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.234.598 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.236.342 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 792 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.249.770 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 224, total running count: 692 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.250.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.251.845 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 793 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.265.182 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 225, total running count: 693 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.265.541 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.267.211 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 794 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.280.715 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 226, total running count: 694 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.281.056 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.282.592 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 795 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.296.324 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 227, total running count: 695 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.296.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.298.965 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 796 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.311.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 228, total running count: 696 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.312.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.314.274 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 797 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.327.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 229, total running count: 697 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.327.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.329.674 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 798 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.342.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 230, total running count: 698 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.343.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.345.244 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 799 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.358.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 231, total running count: 699 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.358.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.360.571 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 800 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.373.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 232, total running count: 700 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.374.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.375.981 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 801 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.389.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 233, total running count: 701 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.389.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.391.454 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 802 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.405.218 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 234, total running count: 702 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.405.583 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.407.992 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 803 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.420.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 235, total running count: 703 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.421.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.423.441 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 804 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.435.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 236, total running count: 704 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.436.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.438.819 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 805 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.451.458 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 237, total running count: 705 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.451.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.454.235 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 806 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.466.916 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 238, total running count: 706 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.467.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.469.612 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 807 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.482.586 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 239, total running count: 707 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.482.939 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.484.930 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 808 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.498.027 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 240, total running count: 708 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.498.388 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.500.390 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 809 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.513.551 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 241, total running count: 709 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.513.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.515.829 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 810 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.528.885 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 242, total running count: 710 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.529.277 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.531.259 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 811 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.544.665 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 243, total running count: 711 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.545.039 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.546.610 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 812 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.560.317 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 244, total running count: 712 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.560.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.563.178 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 813 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.576.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 245, total running count: 713 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.576.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.578.686 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 814 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.591.630 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 246, total running count: 714 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.592.008 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.594.166 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 815 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.607.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 247, total running count: 715 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.607.725 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.609.592 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 816 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.622.936 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 248, total running count: 716 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.623.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.625.072 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 817 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.638.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 249, total running count: 717 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.638.933 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.641.571 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 818 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.654.308 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 250, total running count: 718 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.654.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.657.041 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 819 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.669.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 251, total running count: 719 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.670.270 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.672.422 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 820 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.685.574 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 252, total running count: 720 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.685.948 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.687.861 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 821 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.701.037 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 253, total running count: 721 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.701.388 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.703.249 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 822 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.716.532 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 254, total running count: 722 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.716.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.718.774 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 823 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.731.912 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 255, total running count: 723 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.732.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.734.197 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 824 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.747.251 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 256, total running count: 724 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.747.636 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.749.600 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 825 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.762.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 257, total running count: 725 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.763.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.764.962 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 826 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.778.053 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 258, total running count: 726 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.778.437 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.780.425 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 827 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.793.650 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 259, total running count: 727 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.794.031 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.795.818 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 828 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.809.136 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 260, total running count: 728 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.809.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.811.197 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 829 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.824.562 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 261, total running count: 729 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.824.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.826.652 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 830 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.840.096 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 262, total running count: 730 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.840.453 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.842.141 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 831 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.855.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 263, total running count: 731 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.856.086 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.858.741 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 832 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.871.162 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 264, total running count: 732 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.871.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.874.172 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 833 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.886.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 265, total running count: 733 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.887.002 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.889.611 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 834 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.902.027 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 266, total running count: 734 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.902.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.905.041 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 835 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.917.701 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 267, total running count: 735 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.918.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.920.426 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 836 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.933.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 268, total running count: 736 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:38.933.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.935.739 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 837 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.948.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 269, total running count: 737 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.949.308 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.951.255 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 838 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:38.964.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 270, total running count: 738 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:38.964.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.966.779 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 839 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.979.969 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 271, total running count: 739 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.980.353 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.982.164 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 840 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.995.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 272, total running count: 740 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:38.995.960 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:38.997.606 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 841 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.011.241 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 273, total running count: 741 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.011.599 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.014.218 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 842 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.026.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 274, total running count: 742 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.027.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.029.783 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 843 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.042.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 275, total running count: 743 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.042.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.045.235 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 844 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.058.052 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 276, total running count: 744 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.058.439 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.060.626 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 845 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.073.503 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 277, total running count: 745 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.073.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.076.004 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 846 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.089.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 278, total running count: 746 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.089.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.091.392 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 847 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.104.780 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 279, total running count: 747 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.105.132 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.106.761 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 848 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.120.318 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 280, total running count: 748 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.120.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.123.308 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 849 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.135.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 281, total running count: 749 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.136.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.138.737 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 850 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.151.392 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 282, total running count: 750 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.151.770 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.154.237 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 851 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.166.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 283, total running count: 751 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.167.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.169.653 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 852 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.182.260 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 284, total running count: 752 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.182.617 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.184.991 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 853 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.197.663 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 285, total running count: 753 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.198.056 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.200.360 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 854 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.213.142 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 286, total running count: 754 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.213.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.215.874 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 855 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.228.720 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 287, total running count: 755 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.229.114 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.231.355 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 856 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.244.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 288, total running count: 756 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.244.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.246.781 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 857 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.260.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 289, total running count: 757 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.260.728 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.262.276 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 858 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.275.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 290, total running count: 758 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.276.174 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.277.778 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 859 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.291.481 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 291, total running count: 759 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.291.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.294.352 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 860 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.307.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 292, total running count: 760 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.307.394 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.309.783 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 861 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.322.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 293, total running count: 761 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.322.987 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.325.237 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 862 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.338.039 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 294, total running count: 762 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.338.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.340.603 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 863 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.353.515 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 295, total running count: 763 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.353.971 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.355.964 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 864 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.369.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 296, total running count: 764 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.369.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.371.418 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 865 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.384.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 297, total running count: 765 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.384.859 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.386.790 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 866 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.400.073 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 298, total running count: 766 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.400.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.402.206 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 867 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.415.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 299, total running count: 767 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.415.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.417.662 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 868 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.431.024 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 300, total running count: 768 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.431.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.433.239 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 869 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.446.620 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 301, total running count: 769 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.446.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.448.562 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 870 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.462.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 302, total running count: 770 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.462.645 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.465.031 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 871 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.477.762 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 303, total running count: 771 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.478.153 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.480.503 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 872 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.493.423 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 304, total running count: 772 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.493.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.495.966 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 873 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.508.808 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 305, total running count: 773 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.509.205 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.511.416 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 874 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.524.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 306, total running count: 774 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.524.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.526.802 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 875 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.539.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 307, total running count: 775 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.540.291 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.542.252 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 876 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.555.556 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 308, total running count: 776 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.555.916 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.557.721 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 877 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.571.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 309, total running count: 777 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.571.437 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.573.081 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 878 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.586.588 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 310, total running count: 778 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.586.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.589.550 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 879 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.602.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 311, total running count: 779 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.602.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.604.943 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 880 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.617.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 312, total running count: 780 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.617.985 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.620.349 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 881 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.633.124 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 313, total running count: 781 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.633.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.635.846 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 882 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.648.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 314, total running count: 782 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.649.238 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.651.364 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 883 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.664.439 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 315, total running count: 783 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.664.813 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.666.800 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 884 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.679.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 316, total running count: 784 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.680.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.682.298 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 885 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.695.358 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 317, total running count: 785 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.695.696 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.697.739 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 886 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.710.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 318, total running count: 786 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.711.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.713.093 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 887 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.726.278 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 319, total running count: 787 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.726.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.728.404 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 888 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.741.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 320, total running count: 788 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.742.015 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.743.836 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 889 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.757.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 321, total running count: 789 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.757.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.759.303 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 890 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.772.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 322, total running count: 790 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.773.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.774.690 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 891 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.788.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 323, total running count: 791 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.788.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.790.178 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 892 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.803.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 324, total running count: 792 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.804.035 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.805.547 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 893 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.819.199 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 325, total running count: 793 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.819.556 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.822.189 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 894 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.834.992 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 326, total running count: 794 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.835.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.837.881 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 895 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.850.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 327, total running count: 795 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.851.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.853.577 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 896 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.866.558 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 328, total running count: 796 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.866.930 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.869.339 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 897 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.882.149 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 329, total running count: 797 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.882.554 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.885.023 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 898 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.898.318 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 330, total running count: 798 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.898.677 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.900.895 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 899 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.914.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 331, total running count: 799 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.914.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.916.628 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 900 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.929.674 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 332, total running count: 800 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.930.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.932.326 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 901 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.945.354 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 333, total running count: 801 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:39.945.702 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.948.467 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 902 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.961.182 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 334, total running count: 802 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:39.961.541 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.964.228 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 903 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.976.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 335, total running count: 803 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.977.325 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.979.980 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 904 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:39.992.776 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 336, total running count: 804 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:39.993.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:39.995.593 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 905 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.008.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 337, total running count: 805 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.009.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.011.232 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 906 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.024.453 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 338, total running count: 806 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.024.818 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.026.770 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 907 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.040.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 339, total running count: 807 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.040.651 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.042.432 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 908 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.055.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 340, total running count: 808 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.056.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.058.012 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 909 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.071.678 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 341, total running count: 809 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.072.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.073.912 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 910 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.087.356 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 342, total running count: 810 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.087.728 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.089.710 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 911 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.103.207 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 343, total running count: 811 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.103.569 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.105.383 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 912 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.118.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 344, total running count: 812 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.119.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.121.003 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 913 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.135.178 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 345, total running count: 813 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.135.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.137.854 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 914 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.150.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 346, total running count: 814 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.151.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.153.586 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 915 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.166.720 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 347, total running count: 815 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.167.097 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.169.272 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 916 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.182.302 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 348, total running count: 816 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.182.717 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.185.013 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 917 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.197.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 349, total running count: 817 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.198.140 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.200.700 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 918 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.213.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 350, total running count: 818 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.213.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.216.357 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 919 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.229.122 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 351, total running count: 819 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.229.478 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.231.954 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 920 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.244.716 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 352, total running count: 820 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.245.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.247.444 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 921 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.260.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 353, total running count: 821 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.260.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.262.842 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 922 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.276.037 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 354, total running count: 822 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.276.392 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.278.183 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 923 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.291.627 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 355, total running count: 823 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.292.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.293.672 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 924 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.307.233 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 356, total running count: 824 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.307.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.310.171 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 925 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.322.690 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 357, total running count: 825 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.323.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.325.597 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 926 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.338.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 358, total running count: 826 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.338.791 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.340.945 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 927 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.354.116 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 359, total running count: 827 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.354.483 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.356.441 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 928 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.369.752 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 360, total running count: 828 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.370.111 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.372.052 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 929 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.385.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 361, total running count: 829 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.385.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.387.527 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 930 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.400.719 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 362, total running count: 830 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.401.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.403.010 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 931 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.416.120 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 363, total running count: 831 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.416.471 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.418.799 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 932 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.431.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 364, total running count: 832 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.431.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.434.162 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 933 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.447.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 365, total running count: 833 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.447.381 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.449.552 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 934 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.462.709 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 366, total running count: 834 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.463.042 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.464.940 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 935 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.478.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 367, total running count: 835 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.478.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.480.637 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 936 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.493.558 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 368, total running count: 836 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.493.948 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.496.152 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 937 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.509.100 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 369, total running count: 837 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.509.447 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.511.890 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 938 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.524.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 370, total running count: 838 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.525.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.527.402 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 939 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.540.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 371, total running count: 839 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.540.660 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.542.832 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 940 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.555.905 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 372, total running count: 840 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.556.263 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.558.478 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 941 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.571.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 373, total running count: 841 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.572.079 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.573.964 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 942 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.587.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 374, total running count: 842 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.587.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.589.319 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 943 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.602.978 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 375, total running count: 843 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.603.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.606.020 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 944 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.618.485 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 376, total running count: 844 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.618.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.621.312 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 945 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.633.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 377, total running count: 845 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.634.308 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.636.794 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 946 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:40.649.750 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 378, total running count: 846 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.650.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.652.306 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 947 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.665.609 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 379, total running count: 847 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.666.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.667.730 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 948 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.681.146 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 380, total running count: 848 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.681.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.683.193 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 949 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.696.680 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 381, total running count: 849 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.697.048 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.698.605 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 950 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.712.543 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 382, total running count: 850 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.712.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.715.272 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 951 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.728.286 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 383, total running count: 851 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.728.645 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.730.755 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 952 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.743.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 384, total running count: 852 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.744.336 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.746.234 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 953 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.759.433 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 385, total running count: 853 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.759.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.761.836 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 954 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.775.145 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 386, total running count: 854 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.775.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.777.387 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 955 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.790.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 387, total running count: 855 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.791.024 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.793.064 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 956 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.806.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 388, total running count: 856 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.806.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.808.708 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 957 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.821.657 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 389, total running count: 857 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.822.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.824.106 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 958 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.837.618 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 390, total running count: 858 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.838.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.840.672 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 959 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.853.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 391, total running count: 859 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.853.543 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.855.276 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 960 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.868.738 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 392, total running count: 860 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.869.072 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.870.824 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 961 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.884.311 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 393, total running count: 861 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.884.658 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.886.415 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 962 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.899.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 394, total running count: 862 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.900.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.902.014 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 963 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.915.423 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 395, total running count: 863 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.915.758 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.917.900 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 964 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.930.955 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 396, total running count: 864 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:40.931.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.933.293 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 965 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.946.608 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 397, total running count: 865 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.946.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.948.803 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 966 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.962.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 398, total running count: 866 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:40.962.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.964.344 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 967 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:40.977.861 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 399, total running count: 867 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.978.200 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.979.807 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 968 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.993.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 400, total running count: 868 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:40.993.733 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:40.995.268 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 969 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.008.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 401, total running count: 869 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.009.308 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.011.847 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 970 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.024.462 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 402, total running count: 870 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.024.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.027.342 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 971 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.040.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 403, total running count: 871 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.040.508 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.042.889 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 972 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.055.707 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 404, total running count: 872 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.056.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.058.213 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 973 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.071.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 405, total running count: 873 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.071.702 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.073.905 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 974 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.086.951 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 406, total running count: 874 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.087.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.089.421 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 975 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.102.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 407, total running count: 875 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.103.051 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.104.932 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 976 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.118.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 408, total running count: 876 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.118.498 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.120.403 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 977 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.133.815 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 409, total running count: 877 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.134.187 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.135.912 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 978 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.149.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 410, total running count: 878 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.149.484 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.151.407 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 979 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.164.687 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 411, total running count: 879 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.165.033 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.166.935 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 980 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.180.200 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 412, total running count: 880 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.180.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.182.520 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 981 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.195.589 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 413, total running count: 881 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.195.946 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.198.247 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 982 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.211.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 414, total running count: 882 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.211.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.213.054 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 983 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.226.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 415, total running count: 883 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.227.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.228.850 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 984 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.242.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 416, total running count: 884 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.242.545 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.244.327 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 985 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.257.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 417, total running count: 885 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.258.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.259.892 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 986 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.273.433 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 418, total running count: 886 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.273.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.275.716 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 987 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.288.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 419, total running count: 887 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.289.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.291.230 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 988 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.304.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 420, total running count: 888 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.304.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.306.752 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 989 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.320.096 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 421, total running count: 889 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.320.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.322.269 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 990 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.336.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 422, total running count: 890 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.336.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.338.878 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 991 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.351.592 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 423, total running count: 891 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.351.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.354.439 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 992 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.367.124 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 424, total running count: 892 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.367.471 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.369.857 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 993 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.382.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 425, total running count: 893 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.383.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.385.276 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 994 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.398.441 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 426, total running count: 894 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.398.778 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.400.704 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 995 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.413.863 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 427, total running count: 895 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.414.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.416.148 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 996 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.429.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 428, total running count: 896 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.429.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.431.542 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 997 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.444.780 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 429, total running count: 897 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.445.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.446.912 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 998 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.460.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 430, total running count: 898 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.460.800 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.462.393 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 999 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.475.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 431, total running count: 899 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.476.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.477.815 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1000 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.491.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 432, total running count: 900 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.491.574 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.493.218 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1001 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.506.755 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 433, total running count: 901 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.507.156 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.508.685 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1002 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.522.560 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 434, total running count: 902 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.522.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.525.416 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1003 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.538.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 435, total running count: 903 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.538.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.540.816 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1004 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.553.713 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 436, total running count: 904 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.554.058 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.556.568 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1005 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.569.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 437, total running count: 905 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.569.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.572.121 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1006 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.584.989 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 438, total running count: 906 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.585.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.587.705 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1007 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.600.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 439, total running count: 907 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.600.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.603.286 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1008 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.615.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 440, total running count: 908 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.616.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.618.863 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1009 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.631.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 441, total running count: 909 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.631.731 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.634.288 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1010 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.647.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 442, total running count: 910 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.647.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.649.737 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1011 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.662.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 443, total running count: 911 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.663.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.665.135 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1012 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.678.326 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 444, total running count: 912 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.678.661 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.680.619 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1013 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.693.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 445, total running count: 913 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.694.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.696.065 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1014 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.709.466 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 446, total running count: 914 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.709.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.711.864 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1015 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.725.061 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 447, total running count: 915 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.725.432 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.727.317 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1016 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.740.575 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 448, total running count: 916 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.740.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.742.812 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1017 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.756.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 449, total running count: 917 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.756.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.758.376 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1018 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.771.642 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 450, total running count: 918 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.771.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.773.821 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1019 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.787.153 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 451, total running count: 919 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.787.493 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.789.450 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1020 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.802.766 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 452, total running count: 920 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.803.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.804.999 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1021 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.818.483 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 453, total running count: 921 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.818.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.820.393 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1022 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.834.208 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 454, total running count: 922 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.834.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.837.049 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1023 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.850.097 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 455, total running count: 923 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.850.465 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.852.522 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1024 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.865.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 456, total running count: 924 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.865.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.868.029 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1025 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.881.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 457, total running count: 925 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.881.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.883.409 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1026 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.896.752 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 458, total running count: 926 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.897.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.898.915 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1027 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.912.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 459, total running count: 927 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.912.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.914.394 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1028 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.927.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 460, total running count: 928 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:41.928.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.930.888 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1029 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:41.943.530 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 461, total running count: 929 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.943.899 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.946.429 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1030 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.959.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 462, total running count: 930 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.959.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.962.002 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1031 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.974.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 463, total running count: 931 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:41.975.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.977.530 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1032 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.990.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 464, total running count: 932 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:41.990.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:41.992.930 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1033 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.005.975 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 465, total running count: 933 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.006.314 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.008.423 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1034 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.021.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 466, total running count: 934 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.022.001 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.024.007 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1035 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.037.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 467, total running count: 935 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.037.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.039.528 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1036 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.052.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 468, total running count: 936 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:40:42.053.344 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_1 execution count: 2, execution time: 7280.02 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:40:42.053.529 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_1 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:40:42.053.644 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty epoch: 2 step: 468, loss is 2.312579870223999 [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:40:42.054.646 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.h:76] ContinueSend] continue send at the beginning of the epoch [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:40:42.055.766 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:40:42.055.832 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:40:42.055.872 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_1 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.056.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.058.448 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1037 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.071.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 1, total running count: 937 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.072.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.074.066 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1038 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.087.582 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 2, total running count: 938 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.087.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.089.667 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1039 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.103.254 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 3, total running count: 939 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.103.599 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.106.352 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1040 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.118.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 4, total running count: 940 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.119.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.121.774 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1041 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.134.484 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 5, total running count: 941 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.134.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.137.305 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1042 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.149.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 6, total running count: 942 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.150.345 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.152.706 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1043 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.165.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 7, total running count: 943 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.165.951 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.168.099 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1044 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.181.112 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 8, total running count: 944 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.181.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.183.576 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1045 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.196.556 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 9, total running count: 945 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.196.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.198.973 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1046 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.212.102 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 10, total running count: 946 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.212.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.214.584 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1047 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.227.616 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 11, total running count: 947 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.227.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.230.216 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1048 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.243.326 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 12, total running count: 948 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.243.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.245.751 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1049 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.259.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 13, total running count: 949 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.259.650 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.262.275 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1050 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.274.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 14, total running count: 950 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.275.212 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.277.711 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1051 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.290.347 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 15, total running count: 951 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.290.695 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.293.087 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1052 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.305.796 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 16, total running count: 952 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.306.128 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.308.412 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1053 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.321.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 17, total running count: 953 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.321.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.323.831 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1054 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.336.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 18, total running count: 954 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.337.115 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.339.291 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1055 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.352.267 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 19, total running count: 955 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.352.632 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.354.834 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1056 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.367.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 20, total running count: 956 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.368.020 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.370.182 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1057 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.383.106 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 21, total running count: 957 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.383.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.385.535 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1058 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.398.562 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 22, total running count: 958 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.398.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.400.951 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1059 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.414.091 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 23, total running count: 959 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.414.460 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.416.506 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1060 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.429.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 24, total running count: 960 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.430.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.431.948 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1061 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.445.300 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 25, total running count: 961 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.445.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.447.387 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1062 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.460.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 26, total running count: 962 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.461.248 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.462.831 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1063 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.476.195 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 27, total running count: 963 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.476.559 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.478.139 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1064 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.491.738 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 28, total running count: 964 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.492.105 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.494.721 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1065 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.507.205 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 29, total running count: 965 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.507.544 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.510.175 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1066 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.522.794 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 30, total running count: 966 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.523.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.525.518 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1067 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.538.114 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 31, total running count: 967 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.538.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.540.953 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1068 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.553.618 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 32, total running count: 968 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.553.975 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.556.374 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1069 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.569.066 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 33, total running count: 969 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.569.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.571.865 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1070 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.584.508 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 34, total running count: 970 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.584.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.587.332 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1071 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.599.998 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 35, total running count: 971 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.600.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.602.806 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1072 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.615.495 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 36, total running count: 972 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.615.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.618.286 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1073 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.630.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 37, total running count: 973 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.631.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.633.661 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1074 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.646.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 38, total running count: 974 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.646.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.649.052 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1075 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.661.722 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 39, total running count: 975 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.662.084 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.664.517 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1076 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.677.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 40, total running count: 976 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.677.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.679.991 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1077 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.692.813 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 41, total running count: 977 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.693.173 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.695.436 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1078 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.708.329 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 42, total running count: 978 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.708.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.710.819 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1079 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.723.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 43, total running count: 979 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.724.226 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.726.196 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1080 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.739.421 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 44, total running count: 980 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.739.783 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.741.579 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1081 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.754.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 45, total running count: 981 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.755.226 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.756.920 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1082 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.770.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 46, total running count: 982 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.770.816 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.773.518 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1083 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.785.768 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 47, total running count: 983 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.786.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.787.748 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1084 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.801.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 48, total running count: 984 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.801.632 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.803.233 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1085 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.816.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 49, total running count: 985 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.816.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.818.584 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1086 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.831.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 50, total running count: 986 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.832.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.834.058 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1087 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.847.254 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 51, total running count: 987 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.847.587 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.849.628 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1088 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.862.692 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 52, total running count: 988 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.863.054 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.865.012 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1089 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.878.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 53, total running count: 989 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.878.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.880.250 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1090 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.893.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 54, total running count: 990 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.893.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.895.555 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1091 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.909.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 55, total running count: 991 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.909.480 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.911.088 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1092 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.924.703 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 56, total running count: 992 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.925.046 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.926.612 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1093 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.940.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 57, total running count: 993 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.940.629 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.943.290 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1094 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.955.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 58, total running count: 994 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:42.956.163 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.958.780 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1095 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:42.971.241 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 59, total running count: 995 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.971.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.974.179 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1096 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:42.986.575 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 60, total running count: 996 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:42.986.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:42.989.559 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1097 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.002.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 61, total running count: 997 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.002.602 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.004.938 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1098 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.017.930 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 62, total running count: 998 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.018.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.020.283 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1099 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.033.403 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 63, total running count: 999 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.033.777 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.035.526 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1100 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.048.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 64, total running count: 1000 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.049.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.051.036 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1101 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.064.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 65, total running count: 1001 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.064.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.066.332 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1102 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.079.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 66, total running count: 1002 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.080.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.082.890 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1103 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.095.032 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 67, total running count: 1003 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.095.388 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.097.240 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1104 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.110.526 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 68, total running count: 1004 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.110.880 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.112.656 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1105 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.126.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 69, total running count: 1005 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.126.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.127.993 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1106 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.141.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 70, total running count: 1006 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.142.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.144.495 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1107 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.157.250 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 71, total running count: 1007 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.157.618 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.159.806 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1108 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.172.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 72, total running count: 1008 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.173.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.175.253 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1109 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.188.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 73, total running count: 1009 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.188.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.190.693 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1110 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.203.575 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 74, total running count: 1010 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.203.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.206.086 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1111 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.219.131 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 75, total running count: 1011 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.219.486 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.221.525 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1112 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.234.586 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 76, total running count: 1012 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.234.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.236.907 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1113 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.250.204 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 77, total running count: 1013 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.250.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.252.257 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1114 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.265.609 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 78, total running count: 1014 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.265.998 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.267.676 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1115 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.281.107 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 79, total running count: 1015 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.281.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.284.082 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1116 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.296.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 80, total running count: 1016 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.296.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.299.512 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1117 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.312.072 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 81, total running count: 1017 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.312.408 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.314.925 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1118 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.327.748 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 82, total running count: 1018 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.328.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.330.215 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1119 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.343.171 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 83, total running count: 1019 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.343.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.345.716 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1120 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.358.586 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 84, total running count: 1020 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.358.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.361.221 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1121 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.374.013 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 85, total running count: 1021 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.374.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.376.536 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1122 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.389.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 86, total running count: 1022 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.389.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.391.783 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1123 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.404.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 87, total running count: 1023 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.405.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.407.201 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1124 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.420.651 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 88, total running count: 1024 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.420.993 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.422.638 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1125 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.436.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 89, total running count: 1025 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.436.681 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.439.103 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1126 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.451.815 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 90, total running count: 1026 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.452.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.454.546 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1127 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.467.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 91, total running count: 1027 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.467.625 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.469.970 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1128 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.482.694 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 92, total running count: 1028 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.483.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.485.261 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1129 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.498.319 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 93, total running count: 1029 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.498.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.500.575 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1130 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.513.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 94, total running count: 1030 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.514.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.515.949 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1131 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.529.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 95, total running count: 1031 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.529.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.531.276 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1132 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.544.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 96, total running count: 1032 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.545.254 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.547.846 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1133 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.560.398 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 97, total running count: 1033 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.560.758 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.563.349 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1134 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.575.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 98, total running count: 1034 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.576.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.578.852 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1135 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.591.329 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 99, total running count: 1035 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.591.702 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.594.285 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1136 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.607.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 100, total running count: 1036 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.607.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.609.661 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1137 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.622.796 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 101, total running count: 1037 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.623.130 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.624.982 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1138 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.638.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 102, total running count: 1038 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.638.612 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.640.469 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1139 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.653.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 103, total running count: 1039 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.653.883 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.656.043 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1140 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.668.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 104, total running count: 1040 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.669.317 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.671.557 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1141 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.684.485 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 105, total running count: 1041 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.684.827 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.686.955 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1142 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.699.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 106, total running count: 1042 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.700.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.702.350 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1143 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.715.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 107, total running count: 1043 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.715.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.717.729 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1144 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.730.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 108, total running count: 1044 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.731.345 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.733.202 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1145 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.746.688 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 109, total running count: 1045 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.747.048 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.748.633 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1146 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.762.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 110, total running count: 1046 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.762.522 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.764.033 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1147 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.777.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 111, total running count: 1047 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.777.985 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.780.556 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1148 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.793.039 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 112, total running count: 1048 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.793.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.795.904 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1149 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.808.831 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 113, total running count: 1049 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.809.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.811.284 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1150 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.824.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 114, total running count: 1050 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.824.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.826.775 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1151 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.839.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 115, total running count: 1051 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.839.992 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.842.285 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1152 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.855.053 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 116, total running count: 1052 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.855.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.857.748 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1153 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.870.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 117, total running count: 1053 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.870.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.873.231 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1154 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.886.028 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 118, total running count: 1054 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.886.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.888.615 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1155 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.901.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 119, total running count: 1055 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.901.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.903.958 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1156 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.916.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 120, total running count: 1056 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:43.917.035 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.919.477 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1157 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.932.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 121, total running count: 1057 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.932.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.934.798 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1158 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.947.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 122, total running count: 1058 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.948.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.951.488 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1159 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.964.007 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 123, total running count: 1059 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.965.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.968.008 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1160 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.980.616 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 124, total running count: 1060 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:43.980.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.983.377 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1161 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:43.996.217 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 125, total running count: 1061 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:43.996.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:43.998.879 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1162 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.011.928 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 126, total running count: 1062 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.012.329 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.014.309 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1163 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.027.478 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 127, total running count: 1063 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.027.820 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.029.716 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1164 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.042.844 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 128, total running count: 1064 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.043.199 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.045.172 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1165 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.058.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 129, total running count: 1065 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.058.860 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.060.493 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1166 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.073.879 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 130, total running count: 1066 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.074.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.075.886 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1167 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.089.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 131, total running count: 1067 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.089.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.091.351 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1168 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.104.806 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 132, total running count: 1068 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.105.142 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.106.662 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1169 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.120.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 133, total running count: 1069 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.120.560 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.123.229 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1170 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.135.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 134, total running count: 1070 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.136.039 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.138.683 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1171 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.151.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 135, total running count: 1071 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.151.352 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.153.990 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1172 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.166.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 136, total running count: 1072 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.166.800 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.169.362 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1173 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.181.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 137, total running count: 1073 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.182.300 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.184.758 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1174 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.197.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 138, total running count: 1074 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.197.478 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.200.104 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1175 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.212.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 139, total running count: 1075 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.212.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.215.481 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1176 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.227.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 140, total running count: 1076 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.228.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.230.812 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1177 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.243.420 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 141, total running count: 1077 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.243.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.246.243 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1178 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.259.009 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 142, total running count: 1078 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.259.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.261.598 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1179 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.274.503 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 143, total running count: 1079 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.275.210 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.277.050 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1180 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.290.380 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 144, total running count: 1080 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.290.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.292.397 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1181 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.305.777 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 145, total running count: 1081 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.306.379 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.308.895 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1182 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.321.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 146, total running count: 1082 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.321.880 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.324.280 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1183 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.336.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 147, total running count: 1083 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.337.243 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.339.797 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1184 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.352.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 148, total running count: 1084 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.352.715 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.355.266 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1185 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.368.089 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 149, total running count: 1085 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.368.439 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.370.697 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1186 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.383.644 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 150, total running count: 1086 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.383.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.386.074 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1187 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.399.142 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 151, total running count: 1087 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.399.484 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.401.548 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1188 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.414.599 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 152, total running count: 1088 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.416.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.418.176 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1189 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.431.071 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 153, total running count: 1089 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.432.149 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.434.699 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1190 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.447.184 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 154, total running count: 1090 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.447.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.450.177 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1191 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.462.897 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 155, total running count: 1091 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.463.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.465.635 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1192 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.478.657 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 156, total running count: 1092 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.479.168 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.481.095 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1193 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.494.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 157, total running count: 1093 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.494.897 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.496.502 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1194 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.510.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 158, total running count: 1094 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.510.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.512.997 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1195 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.525.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 159, total running count: 1095 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.525.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.528.469 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1196 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.540.968 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 160, total running count: 1096 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.541.319 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.542.869 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1197 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.556.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 161, total running count: 1097 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.560.812 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.562.957 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1198 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.575.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 162, total running count: 1098 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.576.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.578.475 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1199 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.591.479 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 163, total running count: 1099 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.591.964 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.593.942 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1200 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.607.035 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 164, total running count: 1100 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.607.468 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.609.345 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1201 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.622.618 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 165, total running count: 1101 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.623.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.625.742 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1202 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.638.326 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 166, total running count: 1102 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.638.717 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.641.134 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1203 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.654.106 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 167, total running count: 1103 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.654.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.656.621 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1204 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.669.471 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 168, total running count: 1104 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.669.829 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.672.119 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1205 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.684.998 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 169, total running count: 1105 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.685.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.687.526 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1206 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.700.424 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 170, total running count: 1106 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.700.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.702.910 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1207 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.720.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 171, total running count: 1107 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.720.415 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.722.821 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1208 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.735.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 172, total running count: 1108 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.735.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.738.306 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1209 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.751.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 173, total running count: 1109 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.751.714 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.753.681 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1210 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.766.725 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 174, total running count: 1110 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.767.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.768.974 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1211 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.782.091 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 175, total running count: 1111 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.782.421 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.784.415 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1212 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.797.443 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 176, total running count: 1112 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.797.864 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.799.892 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1213 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.813.001 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 177, total running count: 1113 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.813.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.815.296 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1214 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.830.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 178, total running count: 1114 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.830.653 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.833.046 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1215 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.846.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 179, total running count: 1115 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.847.052 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.849.700 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1216 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.862.547 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 180, total running count: 1116 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.862.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.865.053 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1217 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.878.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 181, total running count: 1117 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.878.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.880.413 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1218 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.894.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 182, total running count: 1118 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.894.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.895.916 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1219 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.909.624 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 183, total running count: 1119 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.909.964 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.912.426 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1220 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.926.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 184, total running count: 1120 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.927.395 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.929.147 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1221 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.944.989 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 185, total running count: 1121 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.945.311 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.947.982 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1222 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:44.960.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 186, total running count: 1122 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.960.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.963.371 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1223 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:44.976.248 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 187, total running count: 1123 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:44.976.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.978.835 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1224 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.991.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 188, total running count: 1124 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:44.992.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:44.994.163 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1225 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.007.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 189, total running count: 1125 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.008.020 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.010.642 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1226 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.023.001 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 190, total running count: 1126 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.023.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.025.987 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1227 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.040.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 191, total running count: 1127 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.041.177 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.043.857 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1228 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.056.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 192, total running count: 1128 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.056.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.059.226 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1229 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.072.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 193, total running count: 1129 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.072.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.074.741 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1230 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.087.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 194, total running count: 1130 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.087.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.090.191 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1231 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.102.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 195, total running count: 1131 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.103.265 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.105.606 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1232 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.118.380 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 196, total running count: 1132 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.118.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.120.976 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1233 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.136.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 197, total running count: 1133 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.136.707 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.138.932 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1234 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.153.068 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 198, total running count: 1134 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.153.396 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.155.504 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1235 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.168.853 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 199, total running count: 1135 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.169.225 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.170.905 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1236 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.184.909 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 200, total running count: 1136 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.185.262 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.187.487 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1237 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.200.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 201, total running count: 1137 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.200.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.202.895 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1238 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.216.421 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 202, total running count: 1138 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.216.781 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.219.435 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1239 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.232.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 203, total running count: 1139 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.232.432 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.234.995 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1240 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.254.379 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 204, total running count: 1140 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.254.792 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.257.389 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1241 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.271.868 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 205, total running count: 1141 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.272.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.273.863 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1242 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.287.705 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 206, total running count: 1142 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.288.087 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.290.264 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1243 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.303.704 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 207, total running count: 1143 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.304.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.305.737 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1244 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.319.580 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 208, total running count: 1144 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.319.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.322.399 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1245 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.335.598 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 209, total running count: 1145 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.335.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.337.823 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1246 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.352.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 210, total running count: 1146 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.352.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.354.301 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1247 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.367.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 211, total running count: 1147 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.368.311 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.370.926 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1248 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.383.806 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 212, total running count: 1148 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.384.171 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.386.455 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1249 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.399.567 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 213, total running count: 1149 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.399.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.401.872 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1250 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.415.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 214, total running count: 1150 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.415.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.417.290 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1251 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.431.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 215, total running count: 1151 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.431.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.433.824 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1252 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.447.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 216, total running count: 1152 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.447.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.450.364 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1253 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.463.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 217, total running count: 1153 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.463.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.465.855 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1254 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.479.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 218, total running count: 1154 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.479.443 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.481.243 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1255 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.494.759 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 219, total running count: 1155 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.495.089 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.497.750 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1256 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.510.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 220, total running count: 1156 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.510.811 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.513.124 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1257 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.526.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 221, total running count: 1157 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.526.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.528.495 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1258 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.542.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 222, total running count: 1158 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.542.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.545.020 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1259 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.557.946 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 223, total running count: 1159 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.558.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.560.528 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1260 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.573.878 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 224, total running count: 1160 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.574.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.575.941 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1261 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.589.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 225, total running count: 1161 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.590.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.592.514 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1262 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.605.345 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 226, total running count: 1162 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.605.716 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.607.970 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1263 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.620.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 227, total running count: 1163 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.621.289 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.623.405 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1264 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.636.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 228, total running count: 1164 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.636.856 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.638.832 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1265 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.652.086 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 229, total running count: 1165 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.652.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.654.198 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1266 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.667.645 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 230, total running count: 1166 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.668.000 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.669.525 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1267 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.683.295 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 231, total running count: 1167 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.683.641 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.686.033 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1268 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.699.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 232, total running count: 1168 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.699.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.701.440 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1269 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.714.889 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 233, total running count: 1169 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.715.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.716.842 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1270 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.730.439 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 234, total running count: 1170 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.730.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.733.377 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1271 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.746.066 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 235, total running count: 1171 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.746.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.748.859 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1272 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.761.551 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 236, total running count: 1172 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.761.951 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.764.234 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1273 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.777.252 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 237, total running count: 1173 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.777.608 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.779.676 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1274 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.792.865 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 238, total running count: 1174 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.793.250 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.795.060 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1275 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.808.338 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 239, total running count: 1175 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.808.692 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.810.489 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1276 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.823.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 240, total running count: 1176 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:45.824.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.826.924 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1277 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.839.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 241, total running count: 1177 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.839.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.842.373 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1278 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.855.136 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 242, total running count: 1178 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.855.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.857.796 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1279 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.870.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 243, total running count: 1179 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.870.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.873.120 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1280 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.885.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 244, total running count: 1180 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.886.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.888.585 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1281 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.901.380 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 245, total running count: 1181 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.901.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.904.002 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1282 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.917.171 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 246, total running count: 1182 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.917.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.919.485 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1283 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.932.650 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 247, total running count: 1183 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.932.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.934.932 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1284 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.948.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 248, total running count: 1184 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.948.346 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.950.414 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1285 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.963.817 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 249, total running count: 1185 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.964.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.965.777 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1286 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:45.979.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 250, total running count: 1186 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:45.979.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.981.247 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1287 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.994.942 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 251, total running count: 1187 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:45.995.270 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:45.997.924 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1288 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.010.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 252, total running count: 1188 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.010.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.013.335 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1289 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.026.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 253, total running count: 1189 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.026.661 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.028.739 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1290 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.041.716 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 254, total running count: 1190 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.042.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.044.077 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1291 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.057.115 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 255, total running count: 1191 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.057.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.059.437 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1292 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.072.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 256, total running count: 1192 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.072.880 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.074.875 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1293 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.088.056 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 257, total running count: 1193 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.088.385 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.090.298 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1294 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.104.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 258, total running count: 1194 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.104.533 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.106.840 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1295 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.119.864 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 259, total running count: 1195 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.120.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.122.316 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1296 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.135.243 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 260, total running count: 1196 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.135.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.137.711 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1297 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.150.694 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 261, total running count: 1197 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.151.043 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.153.071 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1298 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.166.020 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 262, total running count: 1198 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.166.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.168.369 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1299 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.181.615 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 263, total running count: 1199 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.182.008 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.183.793 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1300 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.197.124 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 264, total running count: 1200 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.197.514 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.199.304 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1301 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.212.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 265, total running count: 1201 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.213.146 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.214.688 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1302 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.228.202 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 266, total running count: 1202 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.228.533 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.230.186 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1303 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.243.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 267, total running count: 1203 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.243.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.245.536 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1304 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.258.732 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 268, total running count: 1204 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.259.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.260.960 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1305 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.274.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 269, total running count: 1205 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.274.738 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.276.347 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1306 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.289.661 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 270, total running count: 1206 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.290.048 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.291.841 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1307 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.305.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 271, total running count: 1207 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.305.711 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.307.325 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1308 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.320.695 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 272, total running count: 1208 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.321.028 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.322.794 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1309 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.336.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 273, total running count: 1209 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.336.858 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.339.449 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1310 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.352.141 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 274, total running count: 1210 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.352.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.354.940 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1311 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.367.654 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 275, total running count: 1211 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.368.000 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.370.297 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1312 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.383.204 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 276, total running count: 1212 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.383.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.385.646 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1313 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.398.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 277, total running count: 1213 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.399.044 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.400.966 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1314 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.414.079 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 278, total running count: 1214 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.414.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.416.383 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1315 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.429.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 279, total running count: 1215 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.429.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.431.862 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1316 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.444.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 280, total running count: 1216 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.445.290 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.447.335 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1317 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.460.559 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 281, total running count: 1217 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.460.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.462.700 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1318 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.476.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 282, total running count: 1218 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.476.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.478.042 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1319 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.491.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 283, total running count: 1219 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.491.949 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.494.539 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1320 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.507.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 284, total running count: 1220 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.507.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.510.108 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1321 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.522.936 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 285, total running count: 1221 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.523.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.525.491 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1322 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.538.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 286, total running count: 1222 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.538.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.540.721 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1323 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.553.808 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 287, total running count: 1223 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.554.129 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.556.011 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1324 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.569.222 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 288, total running count: 1224 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.569.563 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.571.470 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1325 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.584.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 289, total running count: 1225 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.585.177 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.586.832 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1326 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.600.843 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 290, total running count: 1226 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.601.205 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.603.173 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1327 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.616.164 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 291, total running count: 1227 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.616.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.618.630 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1328 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.631.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 292, total running count: 1228 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.631.897 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.633.937 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1329 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.647.008 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 293, total running count: 1229 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.647.361 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.649.268 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1330 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.662.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 294, total running count: 1230 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.662.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.664.610 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1331 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.677.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 295, total running count: 1231 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.678.289 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.680.028 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1332 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.693.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 296, total running count: 1232 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.693.962 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.695.606 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1333 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.709.136 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 297, total running count: 1233 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.709.493 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.711.114 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1334 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.724.478 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 298, total running count: 1234 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.724.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.726.550 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1335 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.739.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 299, total running count: 1235 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.740.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.741.978 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1336 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.755.638 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 300, total running count: 1236 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.756.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.758.344 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1337 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.771.399 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 301, total running count: 1237 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.771.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.773.782 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1338 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.786.978 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 302, total running count: 1238 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.787.318 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.789.247 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1339 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.802.442 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 303, total running count: 1239 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.802.796 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.804.638 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1340 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.818.031 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 304, total running count: 1240 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.818.370 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.820.008 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1341 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.833.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 305, total running count: 1241 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.833.915 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.836.612 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1342 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.849.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 306, total running count: 1242 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.849.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.852.039 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1343 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.864.751 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 307, total running count: 1243 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.865.109 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.867.433 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1344 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.880.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 308, total running count: 1244 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.880.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.882.831 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1345 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.895.998 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 309, total running count: 1245 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.896.350 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.898.171 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1346 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.911.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 310, total running count: 1246 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.911.793 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.913.563 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1347 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.927.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 311, total running count: 1247 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:46.927.443 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.930.078 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1348 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.942.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 312, total running count: 1248 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.943.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.945.415 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1349 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.958.219 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 313, total running count: 1249 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.958.586 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.960.707 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1350 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:46.973.903 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 314, total running count: 1250 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.974.251 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.976.049 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1351 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:46.989.295 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 315, total running count: 1251 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:46.989.633 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:46.991.610 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1352 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.004.858 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 316, total running count: 1252 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.005.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.006.889 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1353 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.020.782 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 317, total running count: 1253 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.021.139 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.023.679 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1354 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.036.412 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 318, total running count: 1254 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.036.780 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.039.255 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1355 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.051.876 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 319, total running count: 1255 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.052.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.054.675 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1356 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.067.453 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 320, total running count: 1256 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.067.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.070.045 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1357 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.083.059 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 321, total running count: 1257 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.083.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.085.526 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1358 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.098.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 322, total running count: 1258 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.099.165 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.100.889 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1359 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.114.417 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 323, total running count: 1259 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.114.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.117.509 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1360 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.130.061 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 324, total running count: 1260 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.130.416 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.132.948 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1361 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.145.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 325, total running count: 1261 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.146.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.148.449 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1362 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.161.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 326, total running count: 1262 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.161.966 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.164.056 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1363 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.177.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 327, total running count: 1263 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.177.789 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.179.843 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1364 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:47.193.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 328, total running count: 1264 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.194.755 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.196.634 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1365 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.210.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 329, total running count: 1265 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.210.912 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.213.411 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1366 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.226.544 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 330, total running count: 1266 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.226.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.233.927 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1367 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.242.526 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 331, total running count: 1267 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.242.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.244.851 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1368 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.258.181 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 332, total running count: 1268 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.258.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.260.357 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1369 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.274.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 333, total running count: 1269 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.274.501 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.277.240 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1370 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.290.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 334, total running count: 1270 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.290.455 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.292.874 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1371 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.305.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 335, total running count: 1271 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.306.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.308.559 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1372 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.321.362 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 336, total running count: 1272 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.321.770 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.324.112 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1373 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.337.204 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 337, total running count: 1273 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.337.570 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.339.783 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1374 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.352.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 338, total running count: 1274 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.353.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.355.456 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1375 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.368.560 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 339, total running count: 1275 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.368.944 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.370.875 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1376 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.384.171 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 340, total running count: 1276 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.384.517 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.386.403 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1377 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:47.399.778 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 341, total running count: 1277 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.400.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.402.183 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1378 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.415.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 342, total running count: 1278 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:47.416.011 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.417.831 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1379 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.431.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 343, total running count: 1279 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.431.727 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.433.381 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1380 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.447.027 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 344, total running count: 1280 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.447.387 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.450.178 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1381 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.462.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 345, total running count: 1281 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.463.168 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.465.801 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1382 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.478.746 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 346, total running count: 1282 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.479.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.481.540 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1383 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.494.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 347, total running count: 1283 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.494.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.497.241 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1384 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.510.109 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 348, total running count: 1284 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.510.460 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.512.866 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1385 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.525.612 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 349, total running count: 1285 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.525.997 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.528.522 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1386 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.541.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 350, total running count: 1286 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.542.407 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.544.358 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1387 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.557.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 351, total running count: 1287 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.558.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.560.025 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1388 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.573.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 352, total running count: 1288 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.573.837 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.575.515 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1389 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.589.249 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 353, total running count: 1289 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.589.620 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.592.118 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1390 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.605.011 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 354, total running count: 1290 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.605.354 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.607.730 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1391 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.620.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 355, total running count: 1291 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.620.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.623.309 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1392 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.636.121 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 356, total running count: 1292 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.636.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.638.773 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1393 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.651.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 357, total running count: 1293 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.652.241 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.654.401 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1394 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.667.599 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 358, total running count: 1294 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.667.961 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.669.822 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1395 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.683.249 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 359, total running count: 1295 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.683.837 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.686.608 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1396 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.699.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 360, total running count: 1296 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.699.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.701.156 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1397 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.714.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 361, total running count: 1297 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.715.187 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.716.801 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1398 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.730.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 362, total running count: 1298 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.730.812 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.733.432 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1399 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:47.746.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 363, total running count: 1299 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.746.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.748.861 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1400 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.761.713 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 364, total running count: 1300 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.762.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.764.377 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1401 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.777.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 365, total running count: 1301 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.777.660 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.779.863 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1402 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.792.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 366, total running count: 1302 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.793.241 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.795.264 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1403 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.808.439 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 367, total running count: 1303 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.808.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.810.761 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1404 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.824.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 368, total running count: 1304 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.824.575 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.826.559 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1405 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.839.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 369, total running count: 1305 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.840.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.842.130 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1406 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.855.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 370, total running count: 1306 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.855.872 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.857.669 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1407 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.871.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 371, total running count: 1307 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.871.599 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.873.268 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1408 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.887.207 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 372, total running count: 1308 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.887.583 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.889.974 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1409 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.902.813 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 373, total running count: 1309 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.903.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.905.538 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1410 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.918.295 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 374, total running count: 1310 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.918.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.920.959 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1411 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.933.962 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 375, total running count: 1311 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.934.303 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.936.367 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1412 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.949.584 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 376, total running count: 1312 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:47.949.975 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.951.923 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1413 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.965.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 377, total running count: 1313 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.965.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.967.451 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1414 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.980.775 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 378, total running count: 1314 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.981.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.982.884 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1415 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:47.996.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 379, total running count: 1315 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:47.996.728 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:47.999.435 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1416 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.012.109 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 380, total running count: 1316 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.012.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.014.982 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1417 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.027.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 381, total running count: 1317 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.029.822 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.031.617 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1418 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.045.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 382, total running count: 1318 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.046.402 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.048.490 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1419 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.061.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 383, total running count: 1319 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.062.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.063.863 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1420 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.077.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 384, total running count: 1320 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.077.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.080.475 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1421 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.093.195 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 385, total running count: 1321 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.093.572 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.096.094 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1422 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.108.783 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 386, total running count: 1322 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.109.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.111.481 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1423 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.124.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 387, total running count: 1323 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.124.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.126.927 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1424 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.139.962 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 388, total running count: 1324 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.140.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.142.637 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1425 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.155.790 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 389, total running count: 1325 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.156.136 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.158.160 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1426 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.171.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 390, total running count: 1326 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.171.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.173.828 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1427 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.186.967 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 391, total running count: 1327 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.187.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.189.620 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1428 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.202.799 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 392, total running count: 1328 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.203.151 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.205.120 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1429 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.218.407 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 393, total running count: 1329 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.218.748 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.220.599 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1430 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.234.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 394, total running count: 1330 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.234.394 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.236.058 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1431 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.249.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 395, total running count: 1331 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.250.186 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.252.616 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1432 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.265.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 396, total running count: 1332 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.265.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.268.191 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1433 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.281.254 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 397, total running count: 1333 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.281.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.283.709 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1434 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.296.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 398, total running count: 1334 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.297.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.299.142 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1435 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.312.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 399, total running count: 1335 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.312.692 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.314.841 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1436 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.328.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 400, total running count: 1336 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.328.385 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.330.332 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1437 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.343.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 401, total running count: 1337 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.343.922 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.345.876 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1438 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.359.420 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 402, total running count: 1338 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.359.783 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.362.375 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1439 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.375.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 403, total running count: 1339 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.375.551 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.377.834 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1440 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.390.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 404, total running count: 1340 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.391.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.393.466 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1441 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.406.358 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 405, total running count: 1341 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.406.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.408.988 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1442 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.421.966 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 406, total running count: 1342 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.422.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.424.517 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1443 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.437.508 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 407, total running count: 1343 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.437.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.439.875 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1444 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.452.947 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 408, total running count: 1344 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.453.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.455.319 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1445 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.468.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 409, total running count: 1345 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.468.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.470.870 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1446 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.484.168 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 410, total running count: 1346 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.484.517 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.486.344 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1447 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.499.782 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 411, total running count: 1347 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.500.131 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.501.830 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1448 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.515.356 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 412, total running count: 1348 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.515.714 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.518.444 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1449 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.530.975 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 413, total running count: 1349 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.531.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.533.946 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1450 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.546.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 414, total running count: 1350 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.546.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.549.526 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1451 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.561.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 415, total running count: 1351 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.562.336 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.563.950 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1452 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.577.864 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 416, total running count: 1352 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.578.216 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.580.529 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1453 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.593.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 417, total running count: 1353 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.593.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.595.867 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1454 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.608.878 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 418, total running count: 1354 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.609.251 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.611.398 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1455 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.624.531 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 419, total running count: 1355 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.624.879 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.626.925 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1456 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.640.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 420, total running count: 1356 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.640.592 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.642.325 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1457 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.655.844 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 421, total running count: 1357 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.656.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.657.844 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1458 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.671.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 422, total running count: 1358 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.671.544 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.673.365 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1459 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.686.960 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 423, total running count: 1359 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.687.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.688.868 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1460 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.702.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 424, total running count: 1360 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.703.018 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.704.645 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1461 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.717.993 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 425, total running count: 1361 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.718.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.720.103 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1462 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.733.481 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 426, total running count: 1362 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.733.851 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.735.584 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1463 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.748.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 427, total running count: 1363 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.749.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.751.099 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1464 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.764.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 428, total running count: 1364 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.765.135 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.766.758 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1465 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.780.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 429, total running count: 1365 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.780.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.782.377 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1466 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.796.111 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 430, total running count: 1366 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.796.488 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.798.972 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1467 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.811.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 431, total running count: 1367 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.812.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.814.359 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1468 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.827.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 432, total running count: 1368 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.827.794 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.830.061 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1469 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.843.204 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 433, total running count: 1369 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.843.587 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.845.674 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1470 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.858.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 434, total running count: 1370 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.859.241 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.861.265 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1471 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.874.238 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 435, total running count: 1371 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.874.584 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.876.603 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1472 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.889.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 436, total running count: 1372 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.890.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.891.944 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1473 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.905.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 437, total running count: 1373 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.905.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.908.511 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1474 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.921.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 438, total running count: 1374 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.921.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.924.021 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1475 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.936.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 439, total running count: 1375 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.937.072 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.939.431 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1476 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.952.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 440, total running count: 1376 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.952.658 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.954.906 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1477 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.967.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 441, total running count: 1377 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.968.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.970.530 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1478 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.983.151 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 442, total running count: 1378 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:48.983.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:48.986.011 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1479 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:48.998.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 443, total running count: 1379 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:48.998.969 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.001.464 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1480 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.014.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 444, total running count: 1380 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.014.521 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.016.832 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1481 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.029.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 445, total running count: 1381 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.029.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.032.363 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1482 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.044.982 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 446, total running count: 1382 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.045.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.047.900 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1483 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.060.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 447, total running count: 1383 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.060.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.062.745 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1484 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.075.781 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 448, total running count: 1384 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.076.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.078.048 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1485 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.091.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 449, total running count: 1385 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.091.650 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.093.516 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1486 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.106.707 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 450, total running count: 1386 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.107.046 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.108.993 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1487 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.122.173 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 451, total running count: 1387 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.122.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.124.592 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1488 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.137.640 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 452, total running count: 1388 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.138.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.140.239 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1489 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.152.916 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 453, total running count: 1389 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.153.314 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.155.845 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1490 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.168.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 454, total running count: 1390 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.168.794 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.171.315 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1491 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.183.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 455, total running count: 1391 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.184.145 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.185.745 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1492 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.199.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 456, total running count: 1392 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.199.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.201.268 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1493 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.214.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 457, total running count: 1393 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.214.933 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.216.737 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1494 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.230.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 458, total running count: 1394 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.230.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.232.179 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1495 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.245.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 459, total running count: 1395 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.246.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.247.891 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1496 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.261.783 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 460, total running count: 1396 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.262.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.264.585 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1497 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.277.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 461, total running count: 1397 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.277.863 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.280.048 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1498 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.292.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 462, total running count: 1398 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.293.345 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.295.450 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1499 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.308.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 463, total running count: 1399 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.308.769 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.310.902 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1500 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.324.121 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 464, total running count: 1400 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.324.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.326.420 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1501 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.339.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 465, total running count: 1401 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.339.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.342.003 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1502 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.354.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 466, total running count: 1402 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.355.346 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.357.519 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1503 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.370.619 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 467, total running count: 1403 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.370.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.372.901 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1504 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.386.317 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 468, total running count: 1404 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:40:49.386.693 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_1 execution count: 3, execution time: 7330.67 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:40:49.386.908 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_1 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:40:49.387.039 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty epoch: 3 step: 468, loss is 2.2851076126098633 [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:40:49.388.075 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.h:76] ContinueSend] continue send at the beginning of the epoch [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:40:49.389.257 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:40:49.389.324 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:40:49.389.366 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_1 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.389.860 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.392.261 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1505 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.405.330 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 1, total running count: 1405 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.405.681 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.407.956 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1506 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.420.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 2, total running count: 1406 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.421.371 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.423.215 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1507 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.436.732 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 3, total running count: 1407 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.437.084 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.438.684 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1508 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.452.423 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 4, total running count: 1408 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.452.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.455.416 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1509 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.467.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 5, total running count: 1409 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.468.302 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.470.791 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1510 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.483.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 6, total running count: 1410 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.483.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.486.388 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1511 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.499.195 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 7, total running count: 1411 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.499.575 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.501.955 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1512 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.515.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 8, total running count: 1412 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.515.402 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.517.495 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1513 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.530.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 9, total running count: 1413 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.531.236 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.532.900 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1514 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.546.398 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 10, total running count: 1414 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.546.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.548.522 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1515 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.562.132 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 11, total running count: 1415 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.562.484 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.565.200 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1516 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.577.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 12, total running count: 1416 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.578.020 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.580.685 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1517 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.593.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 13, total running count: 1417 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.593.433 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.596.119 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1518 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.608.582 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 14, total running count: 1418 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.608.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.611.478 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1519 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.624.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 15, total running count: 1419 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.624.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.626.872 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1520 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.639.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 16, total running count: 1420 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.640.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.642.316 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1521 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.655.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 17, total running count: 1421 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.655.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.657.714 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1522 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.670.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 18, total running count: 1422 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.670.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.673.072 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1523 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.685.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 19, total running count: 1423 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.686.295 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.688.547 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1524 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.701.592 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 20, total running count: 1424 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.701.998 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.703.946 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1525 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.717.238 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 21, total running count: 1425 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.717.590 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.719.323 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1526 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.732.657 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 22, total running count: 1426 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.733.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.734.816 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1527 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.747.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 23, total running count: 1427 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.748.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.750.328 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1528 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.763.498 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 24, total running count: 1428 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.763.837 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.765.736 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1529 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.779.027 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 25, total running count: 1429 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.779.400 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.781.036 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1530 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.794.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 26, total running count: 1430 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.794.889 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.796.503 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1531 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.810.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 27, total running count: 1431 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.810.382 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.811.976 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1532 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.825.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 28, total running count: 1432 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.825.758 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.827.456 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1533 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.840.592 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 29, total running count: 1433 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.840.973 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.842.830 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1534 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.855.818 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 30, total running count: 1434 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.856.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.858.149 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1535 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.871.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 31, total running count: 1435 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.871.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.873.580 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1536 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.886.650 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 32, total running count: 1436 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.887.033 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.888.998 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1537 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.902.100 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 33, total running count: 1437 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.902.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.904.412 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1538 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.917.511 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 34, total running count: 1438 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.917.863 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.919.664 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1539 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.932.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 35, total running count: 1439 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.933.147 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.935.126 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1540 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.948.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 36, total running count: 1440 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.948.720 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.950.702 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1541 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.963.702 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 37, total running count: 1441 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.964.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.966.010 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1542 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.979.148 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 38, total running count: 1442 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:49.979.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.981.469 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1543 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:49.994.385 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 39, total running count: 1443 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:49.994.727 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:49.996.869 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1544 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.009.962 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 40, total running count: 1444 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.010.338 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.012.373 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1545 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.025.498 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 41, total running count: 1445 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.025.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.027.836 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1546 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.041.027 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 42, total running count: 1446 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.041.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.043.352 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1547 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.056.400 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 43, total running count: 1447 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.056.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.058.786 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1548 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.071.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 44, total running count: 1448 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.072.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.074.251 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1549 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.087.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 45, total running count: 1449 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.087.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.089.651 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1550 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.102.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 46, total running count: 1450 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.103.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.105.118 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1551 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.118.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 47, total running count: 1451 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.118.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.120.460 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1552 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.133.660 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 48, total running count: 1452 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.134.015 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.135.900 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1553 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.149.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 49, total running count: 1453 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.149.501 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.151.463 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1554 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.164.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 50, total running count: 1454 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.164.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.166.864 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1555 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.180.158 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 51, total running count: 1455 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.180.552 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.182.322 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1556 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.195.815 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 52, total running count: 1456 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.196.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.198.857 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1557 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.211.399 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 53, total running count: 1457 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.211.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.214.191 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1558 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.226.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 54, total running count: 1458 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.227.018 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.228.529 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1559 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.242.018 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 55, total running count: 1459 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.242.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.243.922 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1560 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.257.474 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 56, total running count: 1460 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.257.854 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.259.444 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1561 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.272.949 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 57, total running count: 1461 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.273.286 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.274.999 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1562 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.288.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 58, total running count: 1462 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.288.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.290.567 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1563 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.303.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 59, total running count: 1463 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.304.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.305.969 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1564 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.319.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 60, total running count: 1464 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.319.874 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.321.488 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1565 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.335.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 61, total running count: 1465 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.335.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.338.059 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1566 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.350.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 62, total running count: 1466 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.350.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.352.438 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1567 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.365.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 63, total running count: 1467 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.366.275 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.367.886 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1568 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.381.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 64, total running count: 1468 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.381.899 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.384.434 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1569 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.397.030 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 65, total running count: 1469 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.397.378 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.399.879 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1570 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:50.412.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 66, total running count: 1470 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.412.933 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.415.349 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1571 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.427.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 67, total running count: 1471 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.428.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.430.716 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1572 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.443.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 68, total running count: 1472 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.443.629 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.446.176 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1573 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.458.486 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 69, total running count: 1473 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.458.829 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.460.363 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1574 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.473.850 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 70, total running count: 1474 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.474.195 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.476.842 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1575 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.489.251 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 71, total running count: 1475 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.489.615 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.492.276 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1576 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.504.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 72, total running count: 1476 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.505.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.507.700 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1577 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.520.239 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 73, total running count: 1477 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.520.598 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.523.115 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1578 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.535.688 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 74, total running count: 1478 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.536.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.538.571 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1579 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.551.214 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 75, total running count: 1479 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.552.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.555.231 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1580 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.568.195 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 76, total running count: 1480 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.568.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.570.648 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1581 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.583.799 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 77, total running count: 1481 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.584.136 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.585.985 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1582 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.599.222 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 78, total running count: 1482 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.599.569 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.601.406 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1583 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.614.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 79, total running count: 1483 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.615.033 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.616.928 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1584 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.630.336 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 80, total running count: 1484 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.630.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.632.379 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1585 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.645.749 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 81, total running count: 1485 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.646.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.647.915 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1586 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.661.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 82, total running count: 1486 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.662.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.664.568 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1587 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:50.677.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 83, total running count: 1487 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.677.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.679.929 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1588 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.692.822 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 84, total running count: 1488 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.693.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.695.389 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1589 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.708.434 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 85, total running count: 1489 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.708.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.710.796 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1590 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.723.982 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 86, total running count: 1490 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.724.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.726.136 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1591 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.739.533 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 87, total running count: 1491 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.739.874 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.741.555 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1592 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.754.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 88, total running count: 1492 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.755.263 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.756.931 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1593 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.770.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 89, total running count: 1493 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.770.888 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.773.561 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1594 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.786.078 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 90, total running count: 1494 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.786.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.788.999 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1595 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.801.780 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 91, total running count: 1495 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.802.131 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.804.445 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1596 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.817.583 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 92, total running count: 1496 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.817.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.819.967 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1597 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.833.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 93, total running count: 1497 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.833.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.835.429 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1598 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.848.622 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 94, total running count: 1498 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.848.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.850.847 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1599 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.864.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 95, total running count: 1499 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.864.455 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.866.124 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1600 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.879.809 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 96, total running count: 1500 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.880.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.882.667 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1601 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.895.361 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 97, total running count: 1501 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.895.727 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.898.182 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1602 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.910.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 98, total running count: 1502 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.911.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.913.517 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1603 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.926.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 99, total running count: 1503 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.926.361 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.928.935 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1604 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.941.618 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 100, total running count: 1504 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.942.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.944.375 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1605 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:50.957.174 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 101, total running count: 1505 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.957.541 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.959.791 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1606 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.972.554 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 102, total running count: 1506 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.972.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.975.354 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1607 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:50.988.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 103, total running count: 1507 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:50.988.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:50.990.931 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1608 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.003.617 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 104, total running count: 1508 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.003.973 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.006.313 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1609 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.019.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 105, total running count: 1509 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.019.483 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.021.640 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1610 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.034.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 106, total running count: 1510 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.035.151 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.037.186 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1611 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.050.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 107, total running count: 1511 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.050.781 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.052.637 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1612 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.066.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 108, total running count: 1512 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.066.651 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.069.395 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1613 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.081.701 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 109, total running count: 1513 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.082.072 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.083.702 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1614 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.097.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 110, total running count: 1514 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.097.511 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.099.181 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1615 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.112.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 111, total running count: 1515 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.113.150 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.115.686 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1616 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.128.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 112, total running count: 1516 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.128.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.131.263 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1617 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.143.880 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 113, total running count: 1517 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.144.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.146.653 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1618 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.159.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 114, total running count: 1518 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.159.701 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.162.089 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1619 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.174.793 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 115, total running count: 1519 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.175.136 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.177.574 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1620 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.190.482 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 116, total running count: 1520 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.190.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.193.041 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1621 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.205.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 117, total running count: 1521 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.206.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.208.411 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1622 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.221.370 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 118, total running count: 1522 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.221.740 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.223.951 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1623 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.236.636 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 119, total running count: 1523 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.237.018 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.239.483 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1624 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.252.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 120, total running count: 1524 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.252.960 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.254.926 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1625 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.268.314 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 121, total running count: 1525 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.268.688 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.270.290 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1626 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.283.654 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 122, total running count: 1526 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.284.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.285.706 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1627 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.299.331 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 123, total running count: 1527 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.299.692 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.302.337 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1628 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.314.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 124, total running count: 1528 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.315.244 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.317.753 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1629 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.330.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 125, total running count: 1529 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.330.658 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.333.212 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1630 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.345.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 126, total running count: 1530 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.346.045 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.348.643 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1631 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.360.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 127, total running count: 1531 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.361.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.362.888 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1632 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.376.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 128, total running count: 1532 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.376.598 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.378.335 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1633 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.391.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 129, total running count: 1533 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.392.044 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.393.639 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1634 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.407.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 130, total running count: 1534 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.407.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.409.272 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1635 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:51.422.689 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 131, total running count: 1535 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.423.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.425.828 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1636 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.438.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 132, total running count: 1536 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.438.650 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.441.401 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1637 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.453.890 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 133, total running count: 1537 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.454.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.456.832 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1638 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.469.798 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 134, total running count: 1538 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.470.139 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.472.327 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1639 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.485.329 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 135, total running count: 1539 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.485.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.487.821 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1640 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.500.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 136, total running count: 1540 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.501.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.503.264 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1641 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.516.391 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 137, total running count: 1541 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.516.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.518.645 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1642 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.531.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 138, total running count: 1542 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.532.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.534.113 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1643 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.547.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 139, total running count: 1543 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.547.850 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.549.578 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1644 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.563.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 140, total running count: 1544 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:51.563.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.565.045 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1645 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.578.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 141, total running count: 1545 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.578.739 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.580.470 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1646 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.593.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 142, total running count: 1546 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.594.150 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.596.003 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1647 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.609.444 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 143, total running count: 1547 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.609.844 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.611.583 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1648 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.624.872 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 144, total running count: 1548 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.625.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.626.867 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1649 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.640.349 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 145, total running count: 1549 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.640.689 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.642.339 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1650 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.655.680 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 146, total running count: 1550 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.656.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.657.774 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1651 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.671.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 147, total running count: 1551 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.671.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.673.214 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1652 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.686.624 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 148, total running count: 1552 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.686.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.689.624 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1653 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.702.035 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 149, total running count: 1553 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.702.398 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.703.955 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1654 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.717.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 150, total running count: 1554 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.717.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.720.582 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1655 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.733.163 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 151, total running count: 1555 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.733.535 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.735.987 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1656 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.748.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 152, total running count: 1556 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.748.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.751.383 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1657 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.763.964 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 153, total running count: 1557 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.764.331 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.766.724 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1658 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.779.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 154, total running count: 1558 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.779.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.782.157 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1659 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.794.860 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 155, total running count: 1559 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.795.199 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.797.645 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1660 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.810.304 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 156, total running count: 1560 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.810.654 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.813.101 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1661 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.825.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 157, total running count: 1561 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.826.116 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.828.473 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1662 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.841.044 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 158, total running count: 1562 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.841.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.843.733 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1663 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.856.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 159, total running count: 1563 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.857.171 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.859.234 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1664 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.872.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 160, total running count: 1564 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.872.624 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.874.768 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1665 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.887.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 161, total running count: 1565 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.888.016 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.890.106 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1666 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.903.285 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 162, total running count: 1566 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.903.638 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.905.463 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1667 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.918.596 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 163, total running count: 1567 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.918.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.920.929 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1668 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.934.100 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 164, total running count: 1568 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.934.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.936.420 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1669 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.949.421 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 165, total running count: 1569 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.949.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.951.879 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1670 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.965.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 166, total running count: 1570 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:51.965.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.967.376 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1671 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.980.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 167, total running count: 1571 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:51.980.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.982.900 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1672 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.995.594 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 168, total running count: 1572 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:51.995.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:51.998.359 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1673 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.010.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 169, total running count: 1573 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.011.233 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.013.706 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1674 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.026.210 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 170, total running count: 1574 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.026.559 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.028.921 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1675 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.041.863 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 171, total running count: 1575 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.042.199 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.044.313 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1676 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.057.318 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 172, total running count: 1576 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.057.668 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.059.705 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1677 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.072.861 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 173, total running count: 1577 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.073.200 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.075.190 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1678 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.088.290 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 174, total running count: 1578 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.088.657 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.090.560 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1679 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.103.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 175, total running count: 1579 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.104.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.105.987 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1680 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.119.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 176, total running count: 1580 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.119.521 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.121.378 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1681 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.134.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 177, total running count: 1581 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.134.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.136.777 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1682 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.149.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 178, total running count: 1582 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.150.199 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.152.215 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1683 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:52.165.195 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 179, total running count: 1583 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.165.677 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.167.689 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1684 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.180.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 180, total running count: 1584 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.181.235 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.183.203 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1685 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.196.381 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 181, total running count: 1585 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.196.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.198.707 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1686 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.211.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 182, total running count: 1586 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.212.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.214.197 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1687 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.227.115 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 183, total running count: 1587 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.227.483 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.229.545 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1688 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.242.510 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 184, total running count: 1588 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.242.854 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.244.958 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1689 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.257.853 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 185, total running count: 1589 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.258.199 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.260.284 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1690 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.273.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 186, total running count: 1590 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.273.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.275.829 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1691 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.288.943 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 187, total running count: 1591 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.289.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.291.243 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1692 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.304.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 188, total running count: 1592 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.304.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.306.596 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1693 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.319.905 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 189, total running count: 1593 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.320.247 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.322.217 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1694 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.335.224 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 190, total running count: 1594 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.335.572 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.337.705 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1695 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.350.781 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 191, total running count: 1595 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.351.116 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.353.016 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1696 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.366.184 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 192, total running count: 1596 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.366.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.368.384 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1697 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.381.680 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 193, total running count: 1597 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.382.072 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.383.860 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1698 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.397.176 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 194, total running count: 1598 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.397.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.399.308 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1699 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.412.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 195, total running count: 1599 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.413.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.415.897 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1700 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.428.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 196, total running count: 1600 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.428.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.431.231 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1701 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.444.115 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 197, total running count: 1601 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.444.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.446.725 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1702 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.459.877 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 198, total running count: 1602 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.460.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.462.262 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1703 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.475.567 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 199, total running count: 1603 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.475.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.477.782 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1704 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.490.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 200, total running count: 1604 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.491.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.493.220 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1705 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.506.714 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 201, total running count: 1605 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.507.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.508.633 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1706 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.522.289 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 202, total running count: 1606 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.522.633 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.525.155 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1707 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.537.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 203, total running count: 1607 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.538.148 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.540.464 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1708 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.553.330 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 204, total running count: 1608 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.553.660 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.555.918 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1709 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.568.938 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 205, total running count: 1609 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.569.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.571.461 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1710 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.584.394 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 206, total running count: 1610 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.584.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.586.938 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1711 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.599.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 207, total running count: 1611 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.600.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.602.392 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1712 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.615.615 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 208, total running count: 1612 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.615.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.617.804 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1713 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.631.089 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 209, total running count: 1613 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.631.447 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.633.217 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1714 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.646.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 210, total running count: 1614 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.646.900 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.648.584 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1715 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.662.402 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 211, total running count: 1615 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.662.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.665.360 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1716 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.677.748 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 212, total running count: 1616 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.678.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.679.703 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1717 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.693.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 213, total running count: 1617 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.693.654 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.696.264 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1718 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.708.627 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 214, total running count: 1618 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.708.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.710.722 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1719 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.724.285 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 215, total running count: 1619 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.724.633 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.726.228 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1720 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.739.708 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 216, total running count: 1620 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.740.071 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.741.772 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1721 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.755.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 217, total running count: 1621 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.755.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.757.328 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1722 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.770.620 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 218, total running count: 1622 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.770.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.772.743 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1723 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.786.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 219, total running count: 1623 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.786.479 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.788.250 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1724 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.801.637 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 220, total running count: 1624 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.802.020 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.803.679 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1725 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.817.285 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 221, total running count: 1625 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.817.654 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.820.022 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1726 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.832.689 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 222, total running count: 1626 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.833.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.835.484 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1727 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.848.329 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 223, total running count: 1627 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.848.704 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.850.867 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1728 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.863.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 224, total running count: 1628 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.864.231 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.866.203 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1729 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.879.432 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 225, total running count: 1629 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.879.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.881.746 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1730 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.894.890 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 226, total running count: 1630 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.895.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.897.209 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1731 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.910.611 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 227, total running count: 1631 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.911.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.913.775 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1732 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.926.348 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 228, total running count: 1632 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.926.731 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.929.184 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1733 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.941.718 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 229, total running count: 1633 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.942.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.944.609 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1734 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.957.473 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 230, total running count: 1634 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.957.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.960.058 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1735 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.973.336 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 231, total running count: 1635 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:52.973.740 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.975.408 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1736 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:52.988.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 232, total running count: 1636 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:52.989.287 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:52.990.854 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1737 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.004.539 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 233, total running count: 1637 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.004.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.007.487 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1738 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.019.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 234, total running count: 1638 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.020.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.022.897 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1739 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.035.391 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 235, total running count: 1639 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.035.750 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.038.306 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1740 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.050.759 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 236, total running count: 1640 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.051.134 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.053.803 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1741 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.066.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 237, total running count: 1641 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.066.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.069.295 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1742 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.081.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 238, total running count: 1642 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.082.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.084.750 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1743 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.097.388 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 239, total running count: 1643 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.097.783 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.100.222 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1744 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.112.831 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 240, total running count: 1644 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.113.218 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.115.606 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1745 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.128.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 241, total running count: 1645 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.128.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.130.969 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1746 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.143.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 242, total running count: 1646 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.144.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.146.305 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1747 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.159.484 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 243, total running count: 1647 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.159.851 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.161.801 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1748 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.175.128 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 244, total running count: 1648 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.175.511 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.177.168 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1749 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.190.663 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 245, total running count: 1649 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.191.031 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.193.597 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1750 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.206.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 246, total running count: 1650 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.206.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.209.110 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1751 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.222.033 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 247, total running count: 1651 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.222.378 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.224.457 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1752 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.237.476 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 248, total running count: 1652 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.237.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.239.778 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1753 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.253.058 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 249, total running count: 1653 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.253.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.255.317 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1754 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.268.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 250, total running count: 1654 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.268.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.270.875 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1755 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.283.848 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 251, total running count: 1655 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.284.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.286.511 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1756 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.299.508 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 252, total running count: 1656 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.299.863 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.301.984 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1757 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.314.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 253, total running count: 1657 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.315.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.317.564 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1758 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.330.326 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 254, total running count: 1658 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.330.704 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.332.915 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1759 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.346.048 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 255, total running count: 1659 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.346.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.348.387 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1760 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.361.301 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 256, total running count: 1660 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.361.681 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.363.777 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1761 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.377.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 257, total running count: 1661 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.377.433 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.379.231 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1762 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.392.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 258, total running count: 1662 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.392.790 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.394.601 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1763 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.408.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 259, total running count: 1663 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.408.476 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.410.105 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1764 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.423.795 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 260, total running count: 1664 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.424.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.426.681 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1765 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.439.163 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 261, total running count: 1665 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.439.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.442.148 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1766 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.454.736 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 262, total running count: 1666 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.455.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.457.685 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1767 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.470.314 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 263, total running count: 1667 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.470.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.473.199 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1768 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.485.708 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 264, total running count: 1668 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.486.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.488.631 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1769 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.501.138 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 265, total running count: 1669 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.501.582 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.503.915 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1770 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.516.658 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 266, total running count: 1670 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.517.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.519.441 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1771 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.532.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 267, total running count: 1671 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.532.748 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.534.863 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1772 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.548.183 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 268, total running count: 1672 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.548.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.550.459 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1773 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.563.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 269, total running count: 1673 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.564.176 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.565.987 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1774 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.579.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 270, total running count: 1674 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.579.707 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.581.481 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1775 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.594.859 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 271, total running count: 1675 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.595.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.596.953 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1776 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.610.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 272, total running count: 1676 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.610.632 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.612.378 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1777 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.625.897 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 273, total running count: 1677 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.626.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.627.884 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1778 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.641.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 274, total running count: 1678 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.641.715 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.644.343 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1779 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.657.008 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 275, total running count: 1679 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.657.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.659.883 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1780 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.672.488 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 276, total running count: 1680 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.672.833 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.675.399 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1781 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.687.962 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 277, total running count: 1681 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.688.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.691.024 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1782 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.703.404 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 278, total running count: 1682 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.704.183 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.706.581 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1783 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.719.433 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 279, total running count: 1683 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.721.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.723.226 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1784 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.736.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 280, total running count: 1684 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.736.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.738.657 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1785 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.752.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 281, total running count: 1685 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.752.863 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.755.278 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1786 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.767.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 282, total running count: 1686 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.768.483 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.770.690 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1787 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.783.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 283, total running count: 1687 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.784.231 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.786.205 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1788 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.799.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 284, total running count: 1688 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.799.874 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.801.798 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1789 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.815.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 285, total running count: 1689 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.815.592 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.817.271 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1790 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.830.700 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 286, total running count: 1690 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.831.061 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.832.620 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1791 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.846.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 287, total running count: 1691 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.846.720 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.849.277 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1792 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.861.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 288, total running count: 1692 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.862.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.864.794 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1793 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.877.329 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 289, total running count: 1693 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.877.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.880.295 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1794 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.892.854 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 290, total running count: 1694 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.893.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.895.720 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1795 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.908.616 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 291, total running count: 1695 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.908.982 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.911.128 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1796 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:53.924.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 292, total running count: 1696 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.924.665 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.926.612 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1797 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.939.850 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 293, total running count: 1697 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.940.214 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.942.101 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1798 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.955.372 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 294, total running count: 1698 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:53.955.727 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.957.568 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1799 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.971.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 295, total running count: 1699 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:53.971.382 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.973.010 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1800 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.986.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 296, total running count: 1700 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:53.986.967 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:53.988.524 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1801 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.002.027 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 297, total running count: 1701 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.002.387 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.003.948 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1802 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.017.345 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 298, total running count: 1702 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.017.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.020.625 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1803 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.033.078 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 299, total running count: 1703 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.033.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.036.110 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1804 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.048.687 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 300, total running count: 1704 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.049.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.051.547 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1805 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.064.233 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 301, total running count: 1705 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.064.614 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.066.942 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1806 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.079.960 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 302, total running count: 1706 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.080.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.082.250 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1807 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.095.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 303, total running count: 1707 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.095.701 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.097.789 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1808 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.110.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 304, total running count: 1708 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.111.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.113.219 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1809 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.126.391 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 305, total running count: 1709 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.126.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.128.610 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1810 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.141.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 306, total running count: 1710 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.142.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.143.984 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1811 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.157.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 307, total running count: 1711 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.157.935 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.160.586 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1812 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.172.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 308, total running count: 1712 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.173.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.174.830 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1813 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.188.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 309, total running count: 1713 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.189.046 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.191.289 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1814 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:54.204.255 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 310, total running count: 1714 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.204.746 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.206.838 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1815 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.220.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 311, total running count: 1715 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.220.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.222.258 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1816 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.235.569 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 312, total running count: 1716 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.235.930 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.237.659 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1817 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.250.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 313, total running count: 1717 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.251.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.253.082 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1818 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.266.754 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 314, total running count: 1718 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.267.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.269.721 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1819 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.282.465 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 315, total running count: 1719 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.282.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.285.187 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1820 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.298.205 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 316, total running count: 1720 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.298.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.300.611 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1821 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.314.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 317, total running count: 1721 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.314.413 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.315.979 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1822 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.329.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 318, total running count: 1722 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.329.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.332.519 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1823 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.344.962 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 319, total running count: 1723 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.345.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.346.880 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1824 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.360.510 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 320, total running count: 1724 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.360.889 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.363.453 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1825 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.375.874 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 321, total running count: 1725 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.376.271 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.378.834 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1826 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.391.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 322, total running count: 1726 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.391.705 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.394.243 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1827 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.406.880 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 323, total running count: 1727 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.407.232 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.409.628 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1828 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.422.233 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 324, total running count: 1728 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.422.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.425.059 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1829 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.437.588 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 325, total running count: 1729 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.437.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.440.508 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1830 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.453.248 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 326, total running count: 1730 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.453.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.456.167 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1831 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.468.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 327, total running count: 1731 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.469.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.471.755 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1832 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.484.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 328, total running count: 1732 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.485.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.487.371 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1833 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.500.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 329, total running count: 1733 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.501.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.502.890 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1834 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.516.447 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 330, total running count: 1734 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.516.888 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.519.538 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1835 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.532.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 331, total running count: 1735 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.532.473 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.534.960 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1836 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.547.756 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 332, total running count: 1736 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.548.142 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.549.868 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1837 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.563.516 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 333, total running count: 1737 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.563.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.565.439 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1838 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.579.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 334, total running count: 1738 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.579.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.582.168 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1839 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.594.837 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 335, total running count: 1739 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.595.259 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.597.762 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1840 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.610.692 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 336, total running count: 1740 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.611.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.613.424 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1841 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.626.425 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 337, total running count: 1741 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.626.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.629.021 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1842 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.642.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 338, total running count: 1742 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.642.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.644.555 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1843 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.657.718 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 339, total running count: 1743 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.658.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.659.954 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1844 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.673.379 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 340, total running count: 1744 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.673.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.676.665 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1845 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.689.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 341, total running count: 1745 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.689.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.692.091 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1846 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.704.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 342, total running count: 1746 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.705.653 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.707.485 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1847 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.721.024 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 343, total running count: 1747 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.721.562 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.724.332 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1848 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.736.758 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 344, total running count: 1748 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.737.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.738.930 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1849 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.753.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 345, total running count: 1749 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.753.399 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.755.884 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1850 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.768.728 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 346, total running count: 1750 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.769.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.771.444 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1851 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.784.516 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 347, total running count: 1751 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.784.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.786.982 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1852 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.800.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 348, total running count: 1752 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.800.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.802.608 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1853 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.816.178 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 349, total running count: 1753 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.816.556 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.818.358 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1854 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.831.964 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 350, total running count: 1754 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.833.039 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.834.969 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1855 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.852.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 351, total running count: 1755 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.852.503 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.855.146 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1856 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.869.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 352, total running count: 1756 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.869.572 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.871.808 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1857 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.885.099 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 353, total running count: 1757 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.885.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.887.267 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1858 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.901.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 354, total running count: 1758 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.901.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.903.856 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1859 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.916.964 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 355, total running count: 1759 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.917.314 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.919.414 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1860 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.932.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 356, total running count: 1760 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.933.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.935.903 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1861 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.948.900 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 357, total running count: 1761 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.949.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.951.258 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1862 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.964.755 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 358, total running count: 1762 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.965.116 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.966.731 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1863 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:54.980.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 359, total running count: 1763 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.981.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.983.405 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1864 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:54.996.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 360, total running count: 1764 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:54.996.854 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:54.998.907 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1865 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.012.165 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 361, total running count: 1765 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.012.624 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.014.197 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1866 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.027.930 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 362, total running count: 1766 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.028.883 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.030.793 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1867 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.044.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 363, total running count: 1767 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.044.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.047.480 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1868 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.063.078 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 364, total running count: 1768 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.063.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.065.287 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1869 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.079.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 365, total running count: 1769 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.079.459 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.081.799 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1870 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.094.665 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 366, total running count: 1770 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.095.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.097.201 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1871 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.110.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 367, total running count: 1771 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.110.514 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.112.548 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1872 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.126.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 368, total running count: 1772 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.126.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.128.130 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1873 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.141.735 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 369, total running count: 1773 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.142.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.143.779 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1874 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.157.278 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 370, total running count: 1774 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.157.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.159.400 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1875 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.173.065 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 371, total running count: 1775 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.173.442 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.176.158 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1876 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.188.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 372, total running count: 1776 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.190.791 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.192.915 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1877 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.205.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 373, total running count: 1777 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.206.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.208.537 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1878 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.221.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 374, total running count: 1778 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.222.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.223.943 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1879 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.237.651 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 375, total running count: 1779 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.238.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.240.594 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1880 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.253.291 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 376, total running count: 1780 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.253.733 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.255.933 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1881 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.268.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 377, total running count: 1781 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.269.295 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.271.349 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1882 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.284.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 378, total running count: 1782 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.284.779 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.286.765 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1883 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.300.358 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 379, total running count: 1783 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.300.715 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.303.451 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1884 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.316.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 380, total running count: 1784 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.316.523 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.318.826 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1885 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.331.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 381, total running count: 1785 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.332.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.334.361 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1886 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.347.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 382, total running count: 1786 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.347.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.349.989 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1887 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.363.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 383, total running count: 1787 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.363.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.366.577 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1888 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.379.102 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 384, total running count: 1788 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.379.922 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.382.244 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1889 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.395.465 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 385, total running count: 1789 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.395.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.397.836 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1890 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.413.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 386, total running count: 1790 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.413.899 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.415.632 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1891 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.429.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 387, total running count: 1791 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.430.251 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.432.249 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1892 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.445.636 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 388, total running count: 1792 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.446.007 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.447.871 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1893 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.461.602 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 389, total running count: 1793 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.461.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.464.629 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1894 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.477.447 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 390, total running count: 1794 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.477.865 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.480.536 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1895 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.493.163 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 391, total running count: 1795 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.493.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.496.403 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1896 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.508.905 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 392, total running count: 1796 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.509.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.511.945 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1897 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.524.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 393, total running count: 1797 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.525.226 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.527.414 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1898 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.540.736 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 394, total running count: 1798 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.541.530 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.544.014 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1899 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.557.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 395, total running count: 1799 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.557.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.559.406 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1900 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.573.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 396, total running count: 1800 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.573.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.575.954 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1901 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.588.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 397, total running count: 1801 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.589.570 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.591.345 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1902 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.605.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 398, total running count: 1802 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.605.715 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.607.842 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1903 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.621.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 399, total running count: 1803 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.621.683 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.623.335 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1904 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.637.045 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 400, total running count: 1804 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.637.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.639.998 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1905 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.655.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 401, total running count: 1805 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.655.596 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.657.807 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1906 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.672.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 402, total running count: 1806 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.672.738 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.674.497 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1907 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.688.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 403, total running count: 1807 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.688.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.691.173 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1908 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.704.486 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 404, total running count: 1808 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.704.828 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.706.517 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1909 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.720.550 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 405, total running count: 1809 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.720.899 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.723.349 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1910 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.736.545 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 406, total running count: 1810 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.736.908 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.738.775 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1911 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.752.575 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 407, total running count: 1811 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.752.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.755.429 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1912 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.768.843 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 408, total running count: 1812 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.769.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.770.845 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1913 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.784.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 409, total running count: 1813 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.784.890 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.787.537 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1914 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.800.597 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 410, total running count: 1814 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.800.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.803.018 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1915 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.816.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 411, total running count: 1815 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.816.837 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.819.475 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1916 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.832.207 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 412, total running count: 1816 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.832.576 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.834.953 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1917 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.847.903 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 413, total running count: 1817 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.848.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.850.498 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1918 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.863.692 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 414, total running count: 1818 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:55.864.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.866.307 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1919 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.879.647 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 415, total running count: 1819 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.880.058 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.881.973 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1920 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.895.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 416, total running count: 1820 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.896.177 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.898.778 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1921 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.911.308 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 417, total running count: 1821 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.911.679 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.914.157 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1922 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.927.177 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 418, total running count: 1822 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.927.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.929.673 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1923 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.943.138 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 419, total running count: 1823 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.943.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.945.176 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1924 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.958.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 420, total running count: 1824 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:55.959.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.961.854 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1925 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.974.714 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 421, total running count: 1825 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:55.975.121 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.977.364 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1926 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.990.583 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 422, total running count: 1826 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:55.990.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:55.992.705 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1927 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.006.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 423, total running count: 1827 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.006.675 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.009.318 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1928 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.022.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 424, total running count: 1828 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.022.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.024.670 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1929 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.037.666 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 425, total running count: 1829 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.038.071 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.040.076 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1930 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.053.928 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 426, total running count: 1830 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.054.300 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.056.867 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1931 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.070.087 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 427, total running count: 1831 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.070.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.072.370 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1932 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.086.231 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 428, total running count: 1832 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.086.588 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.089.160 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1933 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.101.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 429, total running count: 1833 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.102.233 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.104.573 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1934 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.117.488 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 430, total running count: 1834 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.117.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.120.008 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1935 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.133.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 431, total running count: 1835 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.133.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.135.464 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1936 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.148.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 432, total running count: 1836 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.148.967 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.150.838 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1937 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.164.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 433, total running count: 1837 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.164.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.166.154 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1938 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.180.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 434, total running count: 1838 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.180.473 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.182.907 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1939 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.195.764 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 435, total running count: 1839 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.196.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.198.488 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1940 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.211.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 436, total running count: 1840 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.212.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.213.966 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1941 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.227.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 437, total running count: 1841 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.227.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.230.657 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1942 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.243.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 438, total running count: 1842 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.243.618 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.246.004 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1943 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.258.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 439, total running count: 1843 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.259.042 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.261.515 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1944 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.274.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 440, total running count: 1844 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.274.713 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.277.219 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1945 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.290.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 441, total running count: 1845 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.290.407 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.292.754 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1946 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.305.717 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 442, total running count: 1846 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.306.079 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.308.097 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1947 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.321.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 443, total running count: 1847 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.321.832 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.324.408 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1948 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.337.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 444, total running count: 1848 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.337.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.339.967 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1949 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.353.326 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 445, total running count: 1849 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.353.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.355.476 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1950 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.369.163 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 446, total running count: 1850 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.369.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.372.058 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1951 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.384.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 447, total running count: 1851 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.385.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.387.446 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1952 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.400.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 448, total running count: 1852 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.401.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.402.955 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1953 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.416.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 449, total running count: 1853 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.416.889 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.419.464 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1954 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.432.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 450, total running count: 1854 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.432.588 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.434.980 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1955 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.447.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 451, total running count: 1855 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.448.302 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.450.556 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1956 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.463.514 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 452, total running count: 1856 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.463.872 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.466.049 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1957 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.478.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 453, total running count: 1857 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.479.325 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.481.468 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1958 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.494.644 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 454, total running count: 1858 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.494.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.496.862 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1959 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.510.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 455, total running count: 1859 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.510.744 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.512.267 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1960 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.525.859 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 456, total running count: 1860 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.526.213 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.527.823 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1961 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.541.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 457, total running count: 1861 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.542.370 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.544.445 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1962 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.557.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 458, total running count: 1862 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.557.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.559.805 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1963 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.573.102 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 459, total running count: 1863 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.573.510 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.575.288 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1964 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.588.762 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 460, total running count: 1864 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.589.145 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.590.702 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1965 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.604.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 461, total running count: 1865 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.604.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.607.249 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1966 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.619.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 462, total running count: 1866 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.620.263 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.622.611 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1967 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.635.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 463, total running count: 1867 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.635.822 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.638.085 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1968 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.651.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 464, total running count: 1868 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.651.486 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.653.586 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1969 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.666.574 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 465, total running count: 1869 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.666.937 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.669.178 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1970 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.682.171 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 466, total running count: 1870 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.682.532 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.684.674 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1971 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.697.533 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 467, total running count: 1871 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.697.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.700.106 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1972 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.713.001 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 468, total running count: 1872 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:40:56.713.403 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_1 execution count: 4, execution time: 7323.89 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:40:56.713.591 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_1 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:40:56.713.732 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty epoch: 4 step: 468, loss is 0.36873090267181396 [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:40:56.714.761 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.h:76] ContinueSend] continue send at the beginning of the epoch [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:40:56.715.958 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:40:56.716.022 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:40:56.716.063 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_1 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.717.091 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.719.163 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1973 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.732.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 1, total running count: 1873 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.732.733 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.734.627 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1974 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.748.028 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 2, total running count: 1874 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.748.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.750.091 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1975 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.763.746 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 3, total running count: 1875 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.764.118 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.766.641 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1976 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.779.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 4, total running count: 1876 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.779.594 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.782.036 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1977 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.794.704 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 5, total running count: 1877 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.795.072 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.797.335 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1978 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.810.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 6, total running count: 1878 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.810.883 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.812.765 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1979 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.826.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 7, total running count: 1879 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.826.530 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.828.219 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1980 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.841.616 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 8, total running count: 1880 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.842.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.843.607 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1981 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.857.402 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 9, total running count: 1881 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.857.833 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.860.121 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1982 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.873.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 10, total running count: 1882 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.873.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.875.644 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1983 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.888.695 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 11, total running count: 1883 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.889.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.891.127 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1984 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.904.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 12, total running count: 1884 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.904.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.906.620 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1985 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.919.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 13, total running count: 1885 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.920.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.922.047 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1986 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.935.218 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 14, total running count: 1886 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.935.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.937.534 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1987 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.950.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 15, total running count: 1887 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.950.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.952.956 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1988 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:56.966.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 16, total running count: 1888 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.966.465 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.968.376 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1989 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:56.981.554 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 17, total running count: 1889 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.981.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.983.896 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1990 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.997.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 18, total running count: 1890 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:56.997.679 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:56.999.350 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1991 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.012.800 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 19, total running count: 1891 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.013.158 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.014.888 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1992 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.028.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 20, total running count: 1892 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.028.526 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.030.446 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1993 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.043.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 21, total running count: 1893 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.043.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.045.787 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1994 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.059.249 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 22, total running count: 1894 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.059.642 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.062.262 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1995 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.074.909 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 23, total running count: 1895 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.075.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.077.709 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1996 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.090.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 24, total running count: 1896 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.090.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.092.997 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1997 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.106.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 25, total running count: 1897 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.106.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.108.404 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1998 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.121.997 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 26, total running count: 1898 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.122.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.123.938 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1999 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.137.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 27, total running count: 1899 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.137.987 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.140.560 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2000 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.153.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 28, total running count: 1900 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.153.750 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.155.914 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2001 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.168.930 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 29, total running count: 1901 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.169.270 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.171.362 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2002 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.184.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 30, total running count: 1902 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.184.618 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.186.772 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2003 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.199.778 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 31, total running count: 1903 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.200.124 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.202.137 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2004 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.215.226 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 32, total running count: 1904 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.215.584 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.217.567 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2005 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.230.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 33, total running count: 1905 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.230.900 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.232.932 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2006 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.246.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 34, total running count: 1906 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.246.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.249.564 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2007 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.262.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 35, total running count: 1907 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.262.718 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.265.034 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2008 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.277.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 36, total running count: 1908 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.278.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.280.498 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2009 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.293.738 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 37, total running count: 1909 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.294.118 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.295.938 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2010 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.309.423 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 38, total running count: 1910 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.309.850 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.312.500 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2011 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.325.001 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 39, total running count: 1911 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.325.353 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.327.938 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2012 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:57.340.674 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 40, total running count: 1912 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.341.244 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.343.334 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2013 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.356.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 41, total running count: 1913 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.356.909 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.358.731 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2014 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.372.703 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 42, total running count: 1914 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.373.078 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.375.335 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2015 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.388.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 43, total running count: 1915 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.388.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.390.673 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2016 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.403.854 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 44, total running count: 1916 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.404.200 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.406.108 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2017 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.419.134 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 45, total running count: 1917 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.419.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.421.670 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2018 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.434.678 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 46, total running count: 1918 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.435.043 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.437.016 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2019 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.450.246 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 47, total running count: 1919 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.450.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.452.334 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2020 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.465.818 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 48, total running count: 1920 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.466.183 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.467.769 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2021 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.481.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 49, total running count: 1921 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.481.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.484.330 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2022 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.497.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 50, total running count: 1922 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.497.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.499.750 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2023 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.512.992 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 51, total running count: 1923 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.513.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.515.224 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2024 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.528.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 52, total running count: 1924 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.528.922 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.530.653 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2025 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.544.037 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 53, total running count: 1925 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.544.391 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.546.130 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2026 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.559.530 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 54, total running count: 1926 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.559.885 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.561.532 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2027 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.575.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 55, total running count: 1927 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.575.551 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.577.975 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2028 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.590.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 56, total running count: 1928 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.591.260 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.593.366 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2029 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.606.478 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 57, total running count: 1929 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.606.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.608.699 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2030 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.622.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 58, total running count: 1930 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.622.473 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.624.087 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2031 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.637.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 59, total running count: 1931 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.638.127 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.640.731 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2032 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.653.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 60, total running count: 1932 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.653.811 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.656.105 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2033 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.668.827 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 61, total running count: 1933 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.669.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.671.556 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2034 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.684.503 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 62, total running count: 1934 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.684.871 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.686.921 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2035 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.699.962 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 63, total running count: 1935 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.700.304 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.702.298 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2036 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.715.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 64, total running count: 1936 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.715.748 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.717.857 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2037 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.731.007 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 65, total running count: 1937 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.731.404 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.733.201 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2038 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.747.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 66, total running count: 1938 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.747.562 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.749.791 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2039 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.762.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 67, total running count: 1939 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.763.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.765.256 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2040 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.778.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 68, total running count: 1940 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.778.746 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.780.633 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2041 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.793.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 69, total running count: 1941 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.794.111 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.796.044 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2042 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.809.486 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 70, total running count: 1942 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.809.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.811.609 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2043 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.825.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 71, total running count: 1943 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.825.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.828.019 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2044 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.840.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 72, total running count: 1944 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.841.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.843.556 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2045 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.856.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 73, total running count: 1945 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.856.413 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.858.994 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2046 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.871.657 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 74, total running count: 1946 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.872.007 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.874.290 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2047 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.887.086 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 75, total running count: 1947 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.888.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.889.660 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2048 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.903.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 76, total running count: 1948 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.903.730 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.906.251 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2049 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.919.009 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 77, total running count: 1949 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.919.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.921.571 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2050 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:57.934.905 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 78, total running count: 1950 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.935.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.937.142 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2051 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.950.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 79, total running count: 1951 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.950.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.953.590 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2052 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.966.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 80, total running count: 1952 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.966.458 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.968.949 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2053 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.981.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 81, total running count: 1953 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.981.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.984.283 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2054 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:57.997.112 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 82, total running count: 1954 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:57.997.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:57.999.818 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2055 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.012.701 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 83, total running count: 1955 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.013.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.015.247 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2056 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.028.165 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 84, total running count: 1956 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.028.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.030.727 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2057 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.043.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 85, total running count: 1957 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.043.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.046.191 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2058 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.059.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 86, total running count: 1958 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.059.638 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.061.530 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2059 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.074.903 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 87, total running count: 1959 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.075.269 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.076.879 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2060 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.090.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 88, total running count: 1960 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.090.748 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.092.311 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2061 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.105.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 89, total running count: 1961 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.106.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.107.772 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2062 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.121.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 90, total running count: 1962 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.121.660 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.123.298 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2063 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.136.744 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 91, total running count: 1963 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.137.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.138.728 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2064 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.152.171 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 92, total running count: 1964 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.152.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.154.173 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2065 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.167.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 93, total running count: 1965 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.168.153 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.170.645 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2066 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.183.203 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 94, total running count: 1966 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.183.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.186.038 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2067 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.198.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 95, total running count: 1967 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.199.238 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.201.497 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2068 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.214.184 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 96, total running count: 1968 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.214.559 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.216.964 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2069 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.229.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 97, total running count: 1969 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.230.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.232.410 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2070 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.245.226 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 98, total running count: 1970 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.245.599 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.247.930 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2071 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.260.770 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 99, total running count: 1971 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.261.120 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.263.359 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2072 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.276.165 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 100, total running count: 1972 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.276.517 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.278.749 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2073 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.291.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 101, total running count: 1973 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.292.275 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.294.094 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2074 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.307.649 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 102, total running count: 1974 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.308.007 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.309.577 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2075 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.323.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 103, total running count: 1975 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.323.424 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.326.097 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2076 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.338.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 104, total running count: 1976 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.338.998 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.341.651 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2077 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.354.078 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 105, total running count: 1977 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.354.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.357.134 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2078 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.369.658 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 106, total running count: 1978 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.370.019 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.372.496 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2079 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.385.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 107, total running count: 1979 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.385.465 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.387.908 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2080 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.400.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 108, total running count: 1980 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.400.987 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.403.293 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2081 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.416.225 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 109, total running count: 1981 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.416.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.418.655 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2082 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.431.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 110, total running count: 1982 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.432.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.434.041 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2083 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.447.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 111, total running count: 1983 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.447.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.449.503 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2084 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.462.724 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 112, total running count: 1984 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.463.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.464.896 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2085 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.478.239 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 113, total running count: 1985 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.478.576 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.480.198 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2086 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.493.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 114, total running count: 1986 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.494.262 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.496.812 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2087 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.509.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 115, total running count: 1987 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.509.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.512.210 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2088 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.524.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 116, total running count: 1988 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.524.967 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.527.588 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2089 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.539.989 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 117, total running count: 1989 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.540.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.542.925 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2090 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.555.598 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 118, total running count: 1990 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.555.962 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.558.208 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2091 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.570.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 119, total running count: 1991 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.571.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.573.563 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2092 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.586.277 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 120, total running count: 1992 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.586.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.589.023 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2093 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.601.967 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 121, total running count: 1993 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.602.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.604.369 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2094 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.617.580 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 122, total running count: 1994 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.617.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.619.893 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2095 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.633.134 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 123, total running count: 1995 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.633.476 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.635.265 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2096 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.648.768 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 124, total running count: 1996 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.649.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.650.657 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2097 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.664.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 125, total running count: 1997 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.664.615 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.667.280 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2098 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.680.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 126, total running count: 1998 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.680.425 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.682.923 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2099 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.695.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 127, total running count: 1999 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.696.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.698.291 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2100 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.711.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 128, total running count: 2000 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.711.479 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.713.572 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2101 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.726.574 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 129, total running count: 2001 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.726.915 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.728.893 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2102 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.742.271 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 130, total running count: 2002 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.742.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.744.258 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2103 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.757.739 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 131, total running count: 2003 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.758.080 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.759.654 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2104 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.772.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 132, total running count: 2004 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.773.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.775.090 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2105 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.788.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 133, total running count: 2005 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.788.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.790.507 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2106 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.804.066 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 134, total running count: 2006 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.804.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.807.099 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2107 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.819.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 135, total running count: 2007 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.819.792 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.821.473 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2108 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.834.666 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 136, total running count: 2008 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.835.027 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.836.866 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2109 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.849.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 137, total running count: 2009 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.850.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.852.273 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2110 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.865.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 138, total running count: 2010 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.865.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.867.733 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2111 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.880.788 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 139, total running count: 2011 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.881.120 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.883.193 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2112 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.896.248 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 140, total running count: 2012 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.896.584 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.898.634 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2113 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.912.168 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 141, total running count: 2013 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.912.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.914.064 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2114 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:58.927.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 142, total running count: 2014 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.928.000 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.930.724 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2115 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:58.943.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 143, total running count: 2015 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.943.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.946.090 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2116 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.958.598 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 144, total running count: 2016 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.958.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.961.528 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2117 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.973.937 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 145, total running count: 2017 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.974.267 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.975.883 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2118 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.989.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 146, total running count: 2018 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:58.989.955 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:58.992.428 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2119 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.005.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 147, total running count: 2019 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.005.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.007.826 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2120 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.020.423 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 148, total running count: 2020 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.020.799 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.023.364 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2121 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.035.916 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 149, total running count: 2021 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.036.267 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.038.687 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2122 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.051.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 150, total running count: 2022 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.051.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.054.110 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2123 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.067.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 151, total running count: 2023 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.067.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.069.522 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2124 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.082.832 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 152, total running count: 2024 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.083.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.084.945 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2125 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.098.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 153, total running count: 2025 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.098.674 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.100.368 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2126 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.113.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 154, total running count: 2026 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.114.007 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.115.676 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2127 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.129.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 155, total running count: 2027 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.129.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.131.179 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2128 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.144.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 156, total running count: 2028 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.144.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.146.602 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2129 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.159.961 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 157, total running count: 2029 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.160.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.161.974 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2130 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.175.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 158, total running count: 2030 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.175.951 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.178.534 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2131 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.191.000 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 159, total running count: 2031 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.191.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.193.974 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2132 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.206.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 160, total running count: 2032 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.206.872 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.209.454 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2133 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.222.015 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 161, total running count: 2033 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.222.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.224.932 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2134 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.237.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 162, total running count: 2034 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.237.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.240.377 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2135 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.253.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 163, total running count: 2035 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.253.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.255.916 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2136 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.268.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 164, total running count: 2036 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.268.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.271.313 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2137 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.284.080 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 165, total running count: 2037 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.284.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.286.761 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2138 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.299.681 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 166, total running count: 2038 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.300.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.302.190 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2139 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.315.238 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 167, total running count: 2039 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.315.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.317.658 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2140 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.330.651 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 168, total running count: 2040 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.331.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.333.211 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2141 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.346.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 169, total running count: 2041 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.346.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.348.596 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2142 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.361.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 170, total running count: 2042 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.361.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.363.941 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2143 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.377.121 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 171, total running count: 2043 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.377.468 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.379.367 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2144 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.392.574 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 172, total running count: 2044 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.392.943 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.394.695 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2145 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.408.259 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 173, total running count: 2045 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.408.613 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.411.259 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2146 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.423.733 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 174, total running count: 2046 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:59.424.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.426.595 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2147 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.439.115 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 175, total running count: 2047 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.439.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.441.914 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2148 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.454.511 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 176, total running count: 2048 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.454.885 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.457.308 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2149 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.470.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 177, total running count: 2049 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.470.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.472.569 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2150 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.485.530 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 178, total running count: 2050 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.485.890 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.487.874 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2151 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.500.964 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 179, total running count: 2051 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.501.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.503.466 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2152 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.516.679 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 180, total running count: 2052 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.517.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.518.894 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2153 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.532.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 181, total running count: 2053 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.532.708 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.535.312 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2154 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.548.015 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 182, total running count: 2054 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:40:59.548.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.550.769 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2155 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.563.947 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 183, total running count: 2055 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.564.324 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.566.146 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2156 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.579.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 184, total running count: 2056 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.580.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.581.581 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2157 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.595.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 185, total running count: 2057 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.595.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.598.133 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2158 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.610.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 186, total running count: 2058 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.611.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.613.581 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2159 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.626.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 187, total running count: 2059 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.626.768 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.628.983 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2160 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.642.097 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 188, total running count: 2060 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.642.479 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.644.399 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2161 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.657.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 189, total running count: 2061 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.658.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.659.823 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2162 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.673.523 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 190, total running count: 2062 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.673.878 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.676.347 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2163 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.689.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 191, total running count: 2063 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.689.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.691.844 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2164 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.704.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 192, total running count: 2064 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.704.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.707.207 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2165 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.720.425 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 193, total running count: 2065 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.720.781 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.722.655 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2166 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.735.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 194, total running count: 2066 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.736.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.737.983 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2167 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.751.441 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 195, total running count: 2067 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.751.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.753.353 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2168 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.766.951 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 196, total running count: 2068 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.767.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.769.954 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2169 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.782.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 197, total running count: 2069 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.782.770 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.785.459 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2170 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.797.945 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 198, total running count: 2070 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.798.311 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.800.967 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2171 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.813.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 199, total running count: 2071 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.813.771 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.816.362 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2172 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.828.845 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 200, total running count: 2072 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.829.218 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.831.802 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2173 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.844.361 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 201, total running count: 2073 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.844.702 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.847.196 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2174 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.860.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 202, total running count: 2074 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.860.455 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.862.562 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2175 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.875.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 203, total running count: 2075 [INFO] RUNTIME_FRAMEWORK(164039,fffe1b7460f0,python):2024-01-10-11:40:59.875.856 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.877.972 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2176 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.891.039 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 204, total running count: 2076 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.891.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.893.531 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2177 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.906.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 205, total running count: 2077 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.906.754 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.908.900 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2178 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.921.868 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 206, total running count: 2078 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.922.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.924.417 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2179 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.937.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 207, total running count: 2079 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.937.799 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.939.897 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2180 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.952.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 208, total running count: 2080 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:40:59.953.182 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.955.344 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2181 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.968.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 209, total running count: 2081 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.968.755 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.970.778 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2182 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.984.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 210, total running count: 2082 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:40:59.984.399 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:40:59.986.247 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2183 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:40:59.999.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 211, total running count: 2083 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.000.009 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.001.728 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2184 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.015.225 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 212, total running count: 2084 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.015.569 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.017.153 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2185 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.030.805 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 213, total running count: 2085 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.031.169 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.033.712 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2186 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.046.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 214, total running count: 2086 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.047.045 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.049.164 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2187 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.062.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 215, total running count: 2087 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.062.845 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.064.527 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2188 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.078.128 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 216, total running count: 2088 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.078.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.081.195 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2189 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.093.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 217, total running count: 2089 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.094.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.096.633 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2190 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.109.404 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 218, total running count: 2090 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.109.783 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.112.062 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2191 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.125.112 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 219, total running count: 2091 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.125.465 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.127.446 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2192 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.140.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 220, total running count: 2092 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.140.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.142.877 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2193 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.156.550 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 221, total running count: 2093 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.156.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.159.459 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2194 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.172.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 222, total running count: 2094 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.172.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.174.720 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2195 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.187.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 223, total running count: 2095 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.188.347 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.190.129 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2196 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.203.501 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 224, total running count: 2096 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.203.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.205.558 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2197 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.219.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 225, total running count: 2097 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.219.532 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.222.160 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2198 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.234.651 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 226, total running count: 2098 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.235.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.237.619 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2199 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.250.195 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 227, total running count: 2099 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.250.532 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.252.950 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2200 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.265.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 228, total running count: 2100 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.266.303 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.268.491 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2201 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.281.785 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 229, total running count: 2101 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.282.127 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.283.938 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2202 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.297.437 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 230, total running count: 2102 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.297.860 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.299.424 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2203 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.313.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 231, total running count: 2103 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.313.453 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.315.988 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2204 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.328.713 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 232, total running count: 2104 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.329.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.331.385 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2205 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.344.421 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 233, total running count: 2105 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.344.777 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.346.944 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2206 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.360.114 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 234, total running count: 2106 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.360.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.362.243 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2207 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.375.833 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 235, total running count: 2107 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.376.183 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.378.740 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2208 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.391.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 236, total running count: 2108 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.391.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.394.265 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2209 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.407.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 237, total running count: 2109 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.407.640 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.409.781 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2210 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.422.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 238, total running count: 2110 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.423.213 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.425.246 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2211 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.438.291 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 239, total running count: 2111 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.438.653 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.440.544 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2212 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.453.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 240, total running count: 2112 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.454.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.456.038 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2213 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.469.408 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 241, total running count: 2113 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.469.766 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.471.506 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2214 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.485.042 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 242, total running count: 2114 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.485.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.488.046 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2215 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.500.539 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 243, total running count: 2115 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.500.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.503.421 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2216 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.516.124 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 244, total running count: 2116 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.516.471 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.518.779 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2217 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.531.689 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 245, total running count: 2117 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.532.032 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.534.102 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2218 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.547.186 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 246, total running count: 2118 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.547.527 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.549.371 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2219 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.562.736 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 247, total running count: 2119 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.563.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.564.829 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2220 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.578.304 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 248, total running count: 2120 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.578.636 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.580.265 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2221 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.593.775 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 249, total running count: 2121 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.594.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.595.777 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2222 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.609.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 250, total running count: 2122 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.609.532 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.611.203 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2223 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.624.597 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 251, total running count: 2123 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.624.933 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.626.576 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2224 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.639.966 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 252, total running count: 2124 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.640.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.642.023 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2225 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.655.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 253, total running count: 2125 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.656.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.657.589 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2226 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.670.943 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 254, total running count: 2126 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.671.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.672.922 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2227 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.686.590 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 255, total running count: 2127 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.686.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.689.593 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2228 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.702.222 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 256, total running count: 2128 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.702.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.704.999 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2229 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.717.922 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 257, total running count: 2129 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.718.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.720.460 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2230 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.734.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 258, total running count: 2130 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.734.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.735.870 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2231 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.749.462 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 259, total running count: 2131 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.749.856 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.752.485 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2232 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.764.933 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 260, total running count: 2132 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.765.281 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.767.893 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2233 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.780.837 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 261, total running count: 2133 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.781.173 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.783.310 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2234 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.796.302 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 262, total running count: 2134 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.796.651 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.798.718 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2235 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.812.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 263, total running count: 2135 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.812.335 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.814.123 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2236 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.827.580 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 264, total running count: 2136 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.827.948 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.829.491 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2237 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.843.203 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 265, total running count: 2137 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.843.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.846.041 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2238 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.858.725 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 266, total running count: 2138 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.859.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.861.487 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2239 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.874.479 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 267, total running count: 2139 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.874.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.876.869 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2240 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.889.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 268, total running count: 2140 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.890.318 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.892.357 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2241 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.905.731 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 269, total running count: 2141 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.906.100 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.908.769 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2242 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.921.362 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 270, total running count: 2142 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.921.739 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.924.195 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2243 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.936.827 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 271, total running count: 2143 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.937.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.939.411 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2244 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:00.952.336 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 272, total running count: 2144 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.952.696 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.954.870 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2245 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.967.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 273, total running count: 2145 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.968.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.970.198 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2246 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.983.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 274, total running count: 2146 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.983.848 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:00.985.521 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2247 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:00.999.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 275, total running count: 2147 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:00.999.614 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.002.118 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2248 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.014.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 276, total running count: 2148 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.015.243 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.017.501 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2249 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.030.403 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 277, total running count: 2149 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.030.742 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.032.907 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2250 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.045.850 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 278, total running count: 2150 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.046.195 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.048.466 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2251 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.061.413 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 279, total running count: 2151 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.061.799 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.063.965 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2252 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.076.989 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 280, total running count: 2152 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.077.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.079.395 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2253 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.092.481 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 281, total running count: 2153 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.092.837 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.094.874 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2254 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.107.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 282, total running count: 2154 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.108.347 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.110.269 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2255 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.123.568 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 283, total running count: 2155 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.123.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.125.632 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2256 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.139.091 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 284, total running count: 2156 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.139.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.141.122 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2257 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.154.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 285, total running count: 2157 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.155.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.157.626 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2258 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.170.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 286, total running count: 2158 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.170.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.173.210 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2259 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.185.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 287, total running count: 2159 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.186.350 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.188.543 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2260 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.201.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 288, total running count: 2160 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.201.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.203.944 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2261 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.216.987 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 289, total running count: 2161 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.217.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.219.325 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2262 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.232.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 290, total running count: 2162 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.232.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.234.582 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2263 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.248.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 291, total running count: 2163 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.248.362 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.250.021 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2264 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.263.374 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 292, total running count: 2164 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.263.731 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.265.431 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2265 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.278.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 293, total running count: 2165 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.279.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.280.862 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2266 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.294.627 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 294, total running count: 2166 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.294.992 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.297.408 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2267 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.310.042 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 295, total running count: 2167 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.310.379 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.312.884 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2268 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.325.550 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 296, total running count: 2168 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.325.925 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.328.404 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2269 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.341.154 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 297, total running count: 2169 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.341.493 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.343.774 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2270 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.356.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 298, total running count: 2170 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.356.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.359.272 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2271 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.372.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 299, total running count: 2171 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.372.476 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.374.709 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2272 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.387.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 300, total running count: 2172 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.387.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.390.123 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2273 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.403.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 301, total running count: 2173 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.403.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.405.565 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2274 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.418.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 302, total running count: 2174 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.419.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.420.942 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2275 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.434.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 303, total running count: 2175 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.434.935 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.437.402 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2276 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.450.000 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 304, total running count: 2176 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.450.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.452.781 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2277 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.465.480 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 305, total running count: 2177 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.465.858 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.468.227 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2278 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.480.799 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 306, total running count: 2178 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.481.151 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.483.741 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2279 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.496.186 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 307, total running count: 2179 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.496.526 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.498.075 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2280 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.511.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 308, total running count: 2180 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.511.806 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.513.399 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2281 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.527.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 309, total running count: 2181 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.527.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.530.036 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2282 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.542.423 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 310, total running count: 2182 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.542.778 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.545.407 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2283 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.558.000 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 311, total running count: 2183 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.558.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.560.772 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2284 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.573.262 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 312, total running count: 2184 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.573.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.576.227 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2285 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.588.853 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 313, total running count: 2185 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.589.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.591.616 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2286 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.604.259 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 314, total running count: 2186 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.604.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.607.118 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2287 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.619.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 315, total running count: 2187 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.620.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.622.531 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2288 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.635.112 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 316, total running count: 2188 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.635.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.637.975 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2289 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.650.455 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 317, total running count: 2189 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.650.812 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.653.393 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2290 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.665.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 318, total running count: 2190 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.666.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.668.831 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2291 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.681.182 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 319, total running count: 2191 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.681.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.683.091 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2292 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.696.637 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 320, total running count: 2192 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.696.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.698.597 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2293 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.712.358 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 321, total running count: 2193 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.712.711 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.715.239 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2294 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.727.666 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 322, total running count: 2194 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.728.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.730.656 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2295 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.743.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 323, total running count: 2195 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.743.404 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.746.041 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2296 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.758.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 324, total running count: 2196 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.758.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.761.529 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2297 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.774.120 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 325, total running count: 2197 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.774.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.776.877 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2298 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.789.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 326, total running count: 2198 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.790.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.792.269 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2299 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.805.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 327, total running count: 2199 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.805.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.807.777 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2300 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.820.563 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 328, total running count: 2200 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.820.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.823.193 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2301 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.836.271 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 329, total running count: 2201 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.836.612 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.838.609 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2302 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.851.766 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 330, total running count: 2202 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.852.114 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.854.053 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2303 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.867.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 331, total running count: 2203 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.867.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.869.550 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2304 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.882.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 332, total running count: 2204 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.883.235 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.884.860 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2305 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.898.531 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 333, total running count: 2205 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.898.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.901.443 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2306 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.914.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 334, total running count: 2206 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.914.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.916.818 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2307 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.929.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 335, total running count: 2207 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.930.071 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.932.284 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2308 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.945.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 336, total running count: 2208 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.945.625 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.947.806 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2309 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.960.915 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 337, total running count: 2209 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.961.254 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.963.295 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2310 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.976.503 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 338, total running count: 2210 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:01.976.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.978.654 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2311 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:01.992.385 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 339, total running count: 2211 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:01.992.755 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:01.995.211 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2312 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.007.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 340, total running count: 2212 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.008.361 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.010.688 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2313 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.023.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 341, total running count: 2213 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.023.861 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.026.072 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2314 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.038.925 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 342, total running count: 2214 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.039.275 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.041.535 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2315 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.054.385 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 343, total running count: 2215 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.054.752 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.056.923 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2316 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.069.915 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 344, total running count: 2216 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.070.281 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.072.431 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2317 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.085.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 345, total running count: 2217 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.086.191 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.087.967 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2318 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.101.346 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 346, total running count: 2218 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.101.704 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.103.342 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2319 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.116.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 347, total running count: 2219 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.117.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.119.851 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2320 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.132.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 348, total running count: 2220 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.132.775 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.135.323 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2321 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.147.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 349, total running count: 2221 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.148.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.150.657 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2322 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.163.663 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 350, total running count: 2222 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.164.020 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.166.038 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2323 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.179.251 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 351, total running count: 2223 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.179.590 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.181.419 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2324 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.195.045 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 352, total running count: 2224 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.195.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.198.082 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2325 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.210.599 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 353, total running count: 2225 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.210.939 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.213.567 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2326 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.226.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 354, total running count: 2226 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.226.598 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.228.982 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2327 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.241.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 355, total running count: 2227 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.241.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.244.455 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2328 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.257.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 356, total running count: 2228 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.257.387 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.259.908 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2329 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.272.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 357, total running count: 2229 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.272.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.275.376 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2330 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.288.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 358, total running count: 2230 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.288.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.290.823 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2331 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.303.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 359, total running count: 2231 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.304.145 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.306.235 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2332 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.319.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 360, total running count: 2232 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.319.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.321.549 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2333 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.335.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 361, total running count: 2233 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.335.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.338.044 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2334 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.350.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 362, total running count: 2234 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.350.754 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.353.428 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2335 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.365.661 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 363, total running count: 2235 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.366.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.367.642 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2336 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.381.043 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 364, total running count: 2236 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.381.395 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.383.121 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2337 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.396.839 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 365, total running count: 2237 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.397.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.399.635 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2338 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.412.329 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 366, total running count: 2238 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.412.675 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.415.109 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2339 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.427.809 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 367, total running count: 2239 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.428.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.430.547 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2340 batch(es) to device, channel name: 956bb090-af69-11ee-a425-30fd658829ca [INFO] MD(164039,fffd14ff90f0,python):2024-01-10-11:41:02.430.602 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:596] SendDataToAscend] ExecutionTree finished. Device queue sent number of batches: 2340 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.443.178 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 368, total running count: 2240 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.443.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.458.742 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 369, total running count: 2241 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.459.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.474.345 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 370, total running count: 2242 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.474.691 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.490.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 371, total running count: 2243 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.490.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.505.811 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 372, total running count: 2244 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.506.154 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.521.301 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 373, total running count: 2245 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.521.636 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.536.858 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 374, total running count: 2246 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.537.232 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.552.381 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 375, total running count: 2247 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.552.727 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.567.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 376, total running count: 2248 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.568.318 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.583.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 377, total running count: 2249 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.583.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.599.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 378, total running count: 2250 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.600.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.615.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 379, total running count: 2251 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.615.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.631.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 380, total running count: 2252 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.631.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.647.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 381, total running count: 2253 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.647.372 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.662.789 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 382, total running count: 2254 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.663.127 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.678.394 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 383, total running count: 2255 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.678.727 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.693.832 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 384, total running count: 2256 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.694.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.709.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 385, total running count: 2257 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.709.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.725.044 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 386, total running count: 2258 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.725.398 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.740.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 387, total running count: 2259 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.740.887 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.755.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 388, total running count: 2260 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.756.325 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.771.709 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 389, total running count: 2261 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.772.048 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.787.080 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 390, total running count: 2262 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.787.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.802.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 391, total running count: 2263 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.802.871 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.818.184 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 392, total running count: 2264 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.818.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.833.724 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 393, total running count: 2265 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.834.092 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.849.267 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 394, total running count: 2266 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.849.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.865.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 395, total running count: 2267 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.865.407 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.880.501 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 396, total running count: 2268 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.880.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.895.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 397, total running count: 2269 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.896.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.911.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 398, total running count: 2270 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.911.700 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.928.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 399, total running count: 2271 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.928.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.945.301 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 400, total running count: 2272 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.945.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.961.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 401, total running count: 2273 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.961.503 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:02.976.704 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 402, total running count: 2274 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.977.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:02.992.424 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 403, total running count: 2275 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:02.992.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.008.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 404, total running count: 2276 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.008.709 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.024.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 405, total running count: 2277 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.024.433 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.039.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 406, total running count: 2278 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.039.912 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.055.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 407, total running count: 2279 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.055.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.071.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 408, total running count: 2280 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.071.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.086.425 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 409, total running count: 2281 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.086.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.102.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 410, total running count: 2282 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.102.619 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.117.808 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 411, total running count: 2283 [INFO] RUNTIME_FRAMEWORK(164039,fffe1af450f0,python):2024-01-10-11:41:03.118.239 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.133.354 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 412, total running count: 2284 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.133.764 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.148.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 413, total running count: 2285 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.149.267 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.164.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 414, total running count: 2286 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.164.993 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.179.978 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 415, total running count: 2287 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.180.313 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.195.455 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 416, total running count: 2288 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.195.795 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.210.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 417, total running count: 2289 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.211.226 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.226.251 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 418, total running count: 2290 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.226.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.241.619 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 419, total running count: 2291 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.241.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.257.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 420, total running count: 2292 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.257.398 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.272.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 421, total running count: 2293 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.273.250 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.288.301 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 422, total running count: 2294 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.288.713 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.303.630 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 423, total running count: 2295 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.303.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.319.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 424, total running count: 2296 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.319.361 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.334.239 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 425, total running count: 2297 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.334.599 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.349.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 426, total running count: 2298 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.349.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.364.799 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 427, total running count: 2299 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.365.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.380.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 428, total running count: 2300 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.380.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.395.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 429, total running count: 2301 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.395.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.410.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 430, total running count: 2302 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.411.313 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.426.424 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 431, total running count: 2303 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.426.809 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.441.888 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 432, total running count: 2304 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.442.319 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.460.677 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 433, total running count: 2305 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.461.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.476.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 434, total running count: 2306 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.476.725 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.491.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 435, total running count: 2307 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.492.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.507.567 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 436, total running count: 2308 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.507.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.523.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 437, total running count: 2309 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.523.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.538.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 438, total running count: 2310 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.539.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.554.195 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 439, total running count: 2311 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.554.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.570.016 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 440, total running count: 2312 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.570.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.585.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 441, total running count: 2313 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.586.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.601.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 442, total running count: 2314 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.601.755 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.616.692 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 443, total running count: 2315 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.617.072 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.632.216 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 444, total running count: 2316 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.632.556 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.647.530 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 445, total running count: 2317 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.647.868 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.662.949 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 446, total running count: 2318 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.663.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.678.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 447, total running count: 2319 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.678.758 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.693.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 448, total running count: 2320 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.694.136 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.709.092 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 449, total running count: 2321 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.709.427 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.724.498 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 450, total running count: 2322 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.724.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.739.804 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 451, total running count: 2323 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.740.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.755.412 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 452, total running count: 2324 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.757.326 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.772.348 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 453, total running count: 2325 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.773.032 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.790.394 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 454, total running count: 2326 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.790.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.806.811 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 455, total running count: 2327 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.807.165 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.822.827 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 456, total running count: 2328 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.823.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.838.371 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 457, total running count: 2329 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.838.705 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.853.951 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 458, total running count: 2330 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.854.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.869.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 459, total running count: 2331 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.869.938 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.885.521 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 460, total running count: 2332 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.885.898 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.904.612 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 461, total running count: 2333 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.905.056 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.921.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 462, total running count: 2334 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.921.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.937.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 463, total running count: 2335 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.937.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.953.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 464, total running count: 2336 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.953.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.968.642 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 465, total running count: 2337 [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:03.969.019 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:03.984.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 466, total running count: 2338 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:03.984.916 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:04.001.212 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 467, total running count: 2339 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:04.001.567 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:04.017.054 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 468, total running count: 2340 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:04.017.437 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_1 execution count: 5, execution time: 7301.22 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:04.017.659 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_1 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.017.835 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty epoch: 5 step: 468, loss is 0.07388466596603394 [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.018.768 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.h:76] ContinueSend] continue send at the beginning of the epoch [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.018.909 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.019.103 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.076.346 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at t10k-images-idx3-ubyte. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.076.406 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at t10k-labels-idx1-ubyte. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.076.550 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.078.406 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.078.530 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.078.567 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.078.587 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.078.630 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.078.702 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.078.730 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.078.746 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.078.766 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.078.784 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.078.819 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.078.833 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.078.857 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.078.893 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.079.272 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at t10k-images-idx3-ubyte. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.079.301 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at t10k-labels-idx1-ubyte. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.218.926 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.219.019 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.230.444 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.230.584 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.230.605 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.230.623 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.230.657 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.230.724 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.230.752 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.230.767 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.230.786 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.230.804 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.230.837 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.230.851 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.230.872 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.230.902 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.231.237 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at t10k-images-idx3-ubyte. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.231.264 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at t10k-labels-idx1-ubyte. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.358.690 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.358.785 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.370.122 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.370.243 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.370.264 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.370.281 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.370.314 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.370.382 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.370.408 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.370.423 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.370.444 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.370.462 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.370.495 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.370.524 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.370.545 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.370.576 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.370.892 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] ME(164039:281472963273792,MainProcess):2024-01-10-11:41:04.371.521 [mindspore/dataset/engine/datasets.py:4269] queue_name is newly generated. value is 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.373.282 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.373.378 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.373.398 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.373.413 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.373.442 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.373.506 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.373.531 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.373.556 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.373.577 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.373.594 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.373.627 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.373.640 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.373.659 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.373.709 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.374.010 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.374.191 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1729] InitExecDatasetVm] Start InitDataSet Entry [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:04.374.382 [mindspore/ccsrc/backend/graph_compiler/transform.cc:573] CreateBackend] CreateBackend is: ge [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:04.374.407 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:04.374.421 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEBUG(164039,ffff87fd6440,python):2024-01-10-11:41:04.374.547 [mindspore/ccsrc/debug/debugger/debugger.cc:129] Init] Debugger got device_id: 0 [INFO] DEBUG(164039,ffff87fd6440,python):2024-01-10-11:41:04.374.561 [mindspore/ccsrc/debug/debugger/debugger.cc:131] Init] Debugger got device_target: Ascend [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:04.374.625 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:04.374.639 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.374.650 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1252] SetRunMode] Run graph mode with kernel by kernel by configuration. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:04.374.696 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:546] CompileGraphs] Status record: start compile function graph: _anonymous__377 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.374.785 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unify_mindir_pm_0_erase_invalid_micro_depend in 4.86 us [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:04.374.883 [mindspore/ccsrc/utils/anfalgo.cc:1736] IsNodeOutputDynamicShape] Invalid base shape, node: Default/Return-op0_6 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:04.374.945 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:04.374.960 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:04.374.997 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:04.375.009 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:04.375.031 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:04.375.046 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:04.375.076 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:727] CompileGraph] Compile graph: _anonymous__377, Split segments size: 2 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:04.375.109 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:762] CompileGraphFromSegment] Compile normal segment, the first node: @_anonymous__377:CNode_378{[0]: ValueNode InitDataSetQueue} [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:04.375.171 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:421] CompileGraph] Status record: start compile graph. [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:04.375.201 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:1940] ConstructKernelGraph] Create graph: 2 [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:04.375.287 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:2697] ConstructOutput] Output:@_anonymous__377:CNode_378{[0]: ValueNode InitDataSetQueue} [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.375.552 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:193] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:2 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.375.619 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 8.44 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.375.709 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 65.97 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.375.823 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:154] CommonUnifyMindIR] start common unify mindir opt graph:2 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.375.868 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.375.952 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 77.96 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.375.970 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.014 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 40.26 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.032 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.047 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 14.95 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.061 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.111 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 47.4 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.237 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:79] BackendCommonOptimization] Status record: start common optimization. graph id: 2 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.287 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: add_dynamic_shape_attr [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.343 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_0_add_dynamic_shape_attr in 50.33 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.360 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.402 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_1_convert_dynamic_broadcast_to in 39 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.452 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_2_convert_const_input_to_attr in 29.66 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.491 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_3_custom_op_const_input_to_attr in 18.93 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.527 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 15.7 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.609 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 61.09 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.629 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 1.6 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.677 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_7_flatten_concat_fission in 29.39 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.720 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 22.66 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.758 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_9_broadcast_to_fusion in 19.09 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.798 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 20.49 us [WARNING] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.907 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:92] BackendCommonOptimization] [PROF]backend_common_optimization costs 670 usec. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.376.931 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:99] BackendCommonOptimization] Status record: end common optimization. graph id: 2 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.117 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_0_renorm_split in 36.81 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.140 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.227 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_1_reduce_axis_update in 79.48 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.249 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim HistogramFixedWidth not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.265 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_2_histogram_fixed_width_fusion in 15.37 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.281 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim ClipByNorm not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.298 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_3_clip_by_norm_fission in 18.05 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.315 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.331 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_4_tensor_arry_add_flow_cond1 in 18.02 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.348 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayGather not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.366 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_5_tensor_arry_add_flow_cond2 in 16.75 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.382 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.395 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_6_ge_tensor_arry_cast_index in 11.39 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.408 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArray not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.424 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_7_tensor_array_prepare in 15.12 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.440 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.475 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_8_space_to_batch_nd_attr_update in 31.84 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.493 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.525 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_9_batch_to_space_nd_attr_update in 29.23 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.553 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: avg_pool_grad_unify_mindir [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.596 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_10_avg_pool_grad_unify_mindir in 39.35 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.647 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_11_adam_weight_decay_unify_mindir in 31.34 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.715 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_12_cdist_fission in 24.43 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.764 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_13_cdist_grad_fission in 26.02 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.836 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_14_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 52.46 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.897 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_15_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 36.46 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.941 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_16_sparse_softmax_cross_entropy_with_logits_unify_mindir in 22.06 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.982 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_17_dropout_unify_mindir1 in 20.68 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.377.999 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.046 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_18_dropoutgrad_unify_mindir in 43.53 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.066 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.081 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_19_neighbor_exchange_unify_mindir in 16.18 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.095 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.109 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_20_neighbor_exchange_v2_unify_mindir in 13.2 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.122 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.135 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_21_neighbor_exchange_v2_grad_unify_mindir in 11.87 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.150 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.164 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_22_all_to_all_unify_mindir in 13.67 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.180 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim QuantDTypeCast not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.203 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_23_quant_dtype_cst_adjust in 23.54 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.218 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim FSEDecode not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.231 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_24_fse_decode_adjust in 13.9 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.246 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.259 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_25_bn_split in 13.29 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.271 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.333 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_26_bn_grad_unify_mindir in 58.71 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.351 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.365 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_27_bn_grad_split in 13.76 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.379 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.392 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_28_batchnorm_to_bninfer in 11.79 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.405 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.419 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_29_batchnormgrad_to_bninfergrad in 12.83 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.432 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.447 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_30_batch_norm_grad_infer_fission in 13.38 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.496 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_31_ascend_mindir_op_adapter in 33.15 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.517 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_32_DropoutGenMask in 1.45 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.559 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_33_lamb_fission_ge in 25.48 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.621 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_34_print_insert_placeholder_for_tensor_name in 41.51 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.665 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_35_getnext_for_ge in 24 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.715 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_36_sync_bn_split in 23.62 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.757 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_37_sync_bn_grad_split in 21.52 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.800 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_38_adaptive_max_pool2d_ge_fusion in 22.39 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.840 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_39_avg_pool_grad_for_ge in 19.35 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.864 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:260] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.974 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_40_FlashAttentionFusionV1 in 109 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.378.995 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:374] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.379.106 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_41_FlashAttentionFusionV1 in 108.05 us [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:04.379.479 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:04.379.505 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:04.379.518 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:132] GetRunMode] RunMode::kKernelMode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.379.628 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 69.57 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.379.919 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_0_seed_adapter in 63.29 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.379.946 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: env_op_attr_update [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.379.987 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_1_env_op_attr_update in 36.81 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.380.035 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_2_process call inline in 27.79 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.380.067 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_3_expander_fallback in 12.11 us [WARNING] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.380.148 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:139] GEBackendOptimizeACL] [PROF]ascend_backend_optimize_acl costs 312 usec. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:04.380.185 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] InitDataSetQueue is not defined in opdef. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.380.357 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 9.94 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.380.442 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 51.21 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.380.505 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 37.17 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.380.584 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 55.92 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.380.629 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_4_insert_type_transform_op in 20.3 us [WARNING] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.380.718 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:179] GEBackendOptimizeACLAfterKernelSelect] [PROF]ascend_backend_optimize_acl_after_kernel_select costs 419 usec. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.380.787 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_0_DropoutGenMask in 0.59 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.380.838 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_1_cse in 28.46 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.380.898 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_redundant_op in 38.52 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.380.927 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_3_eliminate_maketuple_getitem in 7.47 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.380.948 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_4_MergeTransData in 1.04 us [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.082 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive InitDataSetQueue [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.177 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive InitDataSetQueue [WARNING] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.197 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:262] CreateKernel] [PROF]create_kernel costs 125 usec. [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.239 [mindspore/ccsrc/backend/common/session/session_basic.cc:1140] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 2 start [INFO] DEBUG(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.254 [mindspore/ccsrc/debug/summary/summary.cc:33] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 2 start [INFO] DEBUG(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.270 [mindspore/ccsrc/debug/summary/summary.cc:54] RecurseSetSummaryNodesForAllGraphs] The total summary nodes is: 0 for graph: 2 [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.318 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:891] PrintGraphExecuteOrder] Graph 2 execution order: [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.364 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[0], node name[Default/InitDataSetQueue-op0_6], logic id[4294967295], stream id[0], node info[@kernel_graph_2:CNode_378{[0]: ValueNode InitDataSetQueue}] [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.394 [mindspore/ccsrc/backend/common/somas/somas.cc:143] Assign] Start Somas Assign for graph 2 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.431 [mindspore/ccsrc/backend/common/somas/somas.cc:151] Assign] Somas Configure success, configuration info: Device Name: Ascend Run by execution order: 1 Enable debug log: 0 Debug log path: . [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.443 [mindspore/ccsrc/backend/common/somas/somas.cc:154] Assign] Start Initialize SOMAS Model [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.487 [mindspore/ccsrc/backend/common/somas/somas.cc:1065] GraphOutputProcess] Set 0 graph output tensors' aligned size to 0. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.530 [mindspore/ccsrc/backend/common/somas/somas.cc:1135] RefNodeProcess] Special Tensor total size: RefNode: input 0 output 0 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.555 [mindspore/ccsrc/backend/common/somas/somas.cc:551] InitSomasModel] No Tensor from graph 2 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.568 [mindspore/ccsrc/backend/common/somas/somas.cc:157] Assign] End Initialize SOMAS Model [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.578 [mindspore/ccsrc/backend/common/somas/somas.cc:160] Assign] No Somas Tensor in graph 2 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.589 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:310] DoSomas] Somas allocate success for graph 2 somas size: 0 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.612 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:801] CreateDeviceAddress] Status record: start create device address. graph id: 2 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.666 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:808] CreateDeviceAddress] Status record: end create device address. graph id: 2 [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.723 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1129] CacheGraphOutputToFrontNodeWithIndex] Get graph backend output nodes. [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.754 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1137] CacheGraphOutputToFrontNodeWithIndex] Get graph front output nodes. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.774 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:507] CompileGraph] Status record: end compile graph. graph id: 2 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.855 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:805] CompileGraphFromSegment] Compile cut segment, the cut node: @_anonymous__377:CNode_379{[0]: ValueNode Return, [1]: CNode_378} [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.922 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:521] Transform] Graph(kernel_graph_2) transforms actor begin, strategy:pipeline_with_execution_order [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:04.381.993 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1232] BuildLoopCountActor] Create loop count actor: kernel_graph_2_LoopCountActor [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:04.382.014 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1265] BuildOutputActor] Create output actor: kernel_graph_2_OutputActor [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:04.382.036 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1285] BuildDataPrepareActor] Create data prepare actor: kernel_graph_2_DataPrepareActor [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:04.382.067 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1518] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_2 start. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:04.382.086 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1520] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_2 end. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:04.382.181 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 1_actor_set_kernel_graph_2_memory_actor_insert in 1.75 us [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:04.382.205 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 2_actor_set_kernel_graph_2_invalid_data_arrow_elimination in 1.05 us [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:04.382.237 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 3_actor_set_kernel_graph_2_multi_actor_fusion in 15.73 us [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:04.382.252 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 4_actor_set_kernel_graph_2_batch_data_arrow_fusion in 0.95 us [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:04.382.272 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:554] Transform] Graph(kernel_graph_2) transforms actor end. [WARNING] VM(164039,ffff87fd6440,python):2024-01-10-11:41:04.382.334 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:612] CompileGraphs] [PROF]compile_func_graph costs 7612 usec. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:04.382.358 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:615] CompileGraphs] Status record: end compile function graph: _anonymous__377, produce actor: kernel_graph_2 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:04.382.388 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_2 [INFO] GE(164039,python):2024-01-10-11:41:04.481.912 [graph_var_manager.cc:1424][EVENT]167202 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:41:04.482.000 [graph_manager.cc:1248][EVENT]167202 PreRun:PreRun start: graph node size 1, session id 31, graph id 30, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:41:04.482.666 [atrace_api.c:28](tid:167202) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:41:04.482.732 [trace_rb_log.c:84](tid:167202) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:41:04.482.749 [atrace_api.c:32](tid:167202) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:41:04.482.778 [client_manager.cpp:157][SetProfilingCallback][tid:167202] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:41:04.483.251 [parallel_partitioner.cc:165][EVENT]167202 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.483.287 [parallel_partitioner.cc:178][EVENT]167202 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [11] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.483.333 [graph_prepare.cc:1378][EVENT]167202 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.483.501 [graph_manager.cc:1050][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [183] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.483.528 [graph_manager.cc:1052][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.483.591 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [1] [INFO] GE(164039,python):2024-01-10-11:41:04.483.624 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.483.699 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [43] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.483.715 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.483.773 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.483.787 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.483.798 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.483.897 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.483.917 [graph_manager.cc:1054][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [374] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.484.172 [graph_manager.cc:1055][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [241] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.484.639 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:41:04.484.660 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.484.671 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.484.680 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferShapePass is [105] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.484.689 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.484.697 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:41:04.484.705 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.484.714 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.484.722 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.485.660 [graph_manager.cc:1056][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [1469] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.485.755 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.485.777 [graph_prepare.cc:1982][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [48] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.485.935 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:41:04.485.954 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [0] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.485.964 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.485.981 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferShapePass is [51] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.485.990 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [5] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.485.999 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:41:04.486.007 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.486.016 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.486.024 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of InferValuePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.486.051 [graph_prepare.cc:1983][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [260] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.486.074 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.486.088 [graph_prepare.cc:1984][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.486.101 [graph_prepare.cc:1985][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.486.115 [graph_prepare.cc:1986][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.486.126 [graph_prepare.cc:1987][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.486.140 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.486.151 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.486.165 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.486.232 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.486.245 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.486.255 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.486.263 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.486.272 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.486.280 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.486.291 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.486.310 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.486.321 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.486.333 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.486.343 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.486.355 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.486.364 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.486.375 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.486.383 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.486.391 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.486.416 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.486.432 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.486.462 [graph_prepare.cc:1988][EVENT]167202 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [328] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.486.478 [graph_manager.cc:1065][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [749] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.499.202 [graph_manager.cc:1077][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12701] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.499.251 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.499.277 [graph_manager.cc:1080][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [39] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.501.533 [graph_manager.cc:1081][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [2242] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.501.573 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.501.587 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.501.601 [graph_manager.cc:1082][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [37] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.501.632 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.501.645 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.501.669 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.501.712 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [32] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.501.727 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.501.739 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.501.751 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.501.785 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [23] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.501.803 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.501.818 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.501.834 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.501.849 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.501.861 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.501.869 [graph_manager.cc:2700][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [243] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.501.948 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.501.961 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.501.970 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.501.979 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.501.987 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.501.995 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CastRemovePass is [8] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.502.004 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.502.012 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.502.020 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.502.029 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.502.037 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [5] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.502.051 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [0] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.502.059 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [0] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.502.068 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.502.076 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.502.086 [graph_manager.cc:2741][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [199] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.095 [graph_manager.cc:2752][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.116 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.127 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.141 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.156 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.167 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.179 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.198 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.211 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.224 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.235 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.250 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.261 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.273 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.285 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.294 [graph_manager.cc:2810][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [181] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.316 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.502.334 [graph_manager.cc:2821][EVENT]167202 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [32] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.362 [graph_manager.cc:1087][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [742] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.492 [graph_manager.cc:1088][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [117] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.526 [graph_manager.cc:1089][EVENT]167202 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.544 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.556 [graph_manager.cc:1097][EVENT]167202 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:41:04.502.576 [graph_manager.cc:3325][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.684 [engine_place.cc:144][EVENT]167202 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.698 [engine_place.cc:144][EVENT]167202 Run:The time cost of aicpu_ascend_kernel::CheckSupported is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.755 [graph_manager.cc:3351][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [167] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.770 [graph_manager.cc:3364][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.831 [engine_partitioner.cc:1139][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.848 [engine_partitioner.cc:1142][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.951 [engine_partitioner.cc:1148][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [94] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.502.976 [engine_partitioner.cc:1155][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.503.012 [engine_partitioner.cc:1164][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [27] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.503.044 [graph_manager.cc:3405][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [262] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.503.061 [graph_manager.cc:3412][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.504.534 [graph_manager.cc:3422][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [1458] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.504.565 [graph_manager.cc:3428][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.504.677 [graph_manager.cc:3467][EVENT]167202 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [93] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.504.705 [graph_manager.cc:3377][EVENT]167202 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [1924] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.504.721 [graph_manager.cc:1106][EVENT]167202 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [2151] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.504.733 [graph_manager.cc:1115][EVENT]167202 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:41:04.504.755 [graph_manager.cc:1130][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.504.786 [graph_manager.cc:1131][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.504.809 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.504.825 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.504.835 [graph_manager.cc:2837][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [34] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.504.883 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.504.895 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.504.904 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of CondRemovePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.504.912 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.504.921 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.504.929 [base_pass.cc:339][EVENT]167202 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:04.504.939 [graph_manager.cc:2864][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [87] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.504.951 [graph_manager.cc:2872][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.504.968 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.504.981 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.504.996 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.009 [compile_nodes_pass.cc:88][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.018 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.035 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.061 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [17] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.083 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.096 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.108 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.119 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.128 [graph_manager.cc:2927][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [163] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.143 [graph_manager.cc:2937][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.166 [graph_manager.cc:2943][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.180 [graph_manager.cc:2950][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.413 [graph_manager.cc:2958][EVENT]167202 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [27] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.441 [graph_manager.cc:1132][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [641] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.533 [graph_manager.cc:1135][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [79] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.572 [graph_manager.cc:2975][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.637 [graph_manager.cc:2981][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [52] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.652 [pass_manager.cc:82][EVENT]167202 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.662 [graph_manager.cc:2986][EVENT]167202 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.671 [graph_manager.cc:1136][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [122] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.754 [graph_manager.cc:3555][EVENT]167202 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [49] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.807 [engine_partitioner.cc:1139][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.823 [engine_partitioner.cc:1142][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.899 [engine_partitioner.cc:1148][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [59] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.920 [engine_partitioner.cc:1155][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.949 [engine_partitioner.cc:1164][EVENT]167202 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.505.968 [graph_builder.cc:865][EVENT]167202 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [183] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.506.028 [graph_builder.cc:288][EVENT]167202 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [44] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.506.116 [graph_builder.cc:293][EVENT]167202 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [75] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.506.296 [model_builder.cc:1133][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [92] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.506.496 [block_mem_assigner.cc:4069][EVENT]172033 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164039,python):2024-01-10-11:41:04.506.496 [block_mem_assigner.cc:4069][EVENT]172031 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164039,python):2024-01-10-11:41:04.506.776 [model_builder.cc:1144][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [460] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.506.800 [model_builder.cc:1152][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.506.815 [model_builder.cc:1157][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.506.923 [model_builder.cc:1167][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [97] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.506.942 [model_builder.cc:1174][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.506.962 [model_builder.cc:1180][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.507.002 [model_builder.cc:1184][EVENT]167202 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [28] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.507.021 [graph_builder.cc:304][EVENT]167202 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [885] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:41:04.507.118 [logger.cc:1071] 167202 ModelBindStream: model_id=576, stream_id=1857, flag=0. [INFO] GE(164039,python):2024-01-10-11:41:04.507.183 [task_generator.cc:804][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.507.233 [task_generator.cc:805][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [36] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.507.699 [task_generator.cc:814][EVENT]167202 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [454] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.507.720 [task_generator.cc:954][EVENT]167202 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [543] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.507.773 [task_generator.cc:967][EVENT]167202 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [30] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:41:04.507.793 [logger.cc:1084] 167202 ModelUnbindStream: model_id=576, stream_id=1857, [INFO] GE(164039,python):2024-01-10-11:41:04.507.850 [graph_builder.cc:310][EVENT]167202 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [816] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.507.955 [graph_manager.cc:1152][EVENT]167202 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2258] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.507.972 [graph_manager.cc:1164][EVENT]167202 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:41:04.508.004 [graph_manager.cc:1271][EVENT]167202 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [24830] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.508.014 [graph_manager.cc:1272][EVENT]167202 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:41:04.508.325 [atrace_api.c:93](tid:167202) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:41:04.508.344 [atrace_api.c:95](tid:167202) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:41:04.508.924 [model_introduction.cc:236][EVENT]167202 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(164039,python):2024-01-10-11:41:04.508.946 [model_introduction.cc:294][EVENT]167202 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(164039,python):2024-01-10-11:41:04.508.959 [model_introduction.cc:366][EVENT]167202 Data:model io_info size:0 [INFO] GE(164039,python):2024-01-10-11:41:04.510.997 [graph_converter.cc:838][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [634] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.511.058 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.511.261 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [188] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.511.319 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [40] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.511.333 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [55] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.511.356 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.511.378 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.511.396 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of ZeroCopy is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.511.428 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CEM is [22] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.511.470 [copy_flow_launch_fuse.cc:395][EVENT]167202 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.511.481 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [41] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.511.498 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.511.516 [base_optimizer.cc:70][EVENT]167202 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.511.537 [graph_converter.cc:849][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [503] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.511.652 [graph_converter.cc:853][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [106] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.512.036 [graph_converter.cc:857][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [371] micro second. [INFO] GE(164039,python):2024-01-10-11:41:04.512.105 [graph_converter.cc:862][EVENT]167202 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [49] micro second. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:04.513.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_2_LoopCountActor) running, loop count: 1, current count: 1, total running count: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:04.513.543 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_2 execution count: 1, execution time: 131.075 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:04.513.631 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_2 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:04.513.829 [mindspore/ccsrc/runtime/device/kernel_runtime_manager.cc:35] ClearGraphResource] Clear device Ascend_0 graph 2 runtime resource [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.516.267 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:198] Compile] Input plan: +-Transfer,send_epoch_end:false,total_batch:0) | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.516.397 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:216] Compile] Plan before optimization: +-Top | +-Transfer,send_epoch_end:false,total_batch:0) | | +-Repeat(count:1) | | | +-Batch(batch_size:32 drop_remainder:true) | | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | | +-MnistDataset [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.516.418 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:60] PrePass] Running pre pass loops. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.516.439 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.516.480 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.516.569 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.516.613 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.516.629 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.516.653 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.516.667 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/cache_transform_pass.cc:182] RunOnTree] Pre pass: Cache transform pass started. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.516.690 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/cache_transform_pass.cc:199] RunOnTree] Pre pass: Cache transform pass complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.516.702 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:91] PrePass] Pre pass offload complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.516.716 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:116] PostPass] Running post pass loops. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.516.748 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:135] PostPass] Post passes complete. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:04.516.784 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:230] Compile] Plan after optimization: +-Top | +-Transfer,send_epoch_end:false,total_batch:0) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:04.517.443 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_data_queue.cc:227] AscendTdtQueue] Select MBUF channel, the capacity of data queue is: 128 [INFO] MD(164039,fffcf3fff0f0,python):2024-01-10-11:41:04.519.924 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at t10k-images-idx3-ubyte. [INFO] MD(164039,fffcf3fff0f0,python):2024-01-10-11:41:04.519.969 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at t10k-labels-idx1-ubyte. [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.527.491 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:456] SendDataToAscend] Device queue, sending data to Ascend. [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.667.129 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:502] SendDataToAscend] Begin to send data to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.667.224 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:1182] PrintBeginInfoWhenFirstBatch] Loading dataset and begin to push first batch into device ... [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.668.075 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:1192] PrintEndInfoWhenFirstBatch] Loading dataset and push first batch into device successful. [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.668.103 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.668.787 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.669.297 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 3 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.669.757 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 4 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.670.203 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 5 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.670.625 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 6 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.671.067 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 7 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.671.491 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 8 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.671.898 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 9 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.672.318 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 10 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.672.720 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 11 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.673.158 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 12 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.673.592 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 13 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.674.041 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 14 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.674.455 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 15 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.674.900 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 16 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.675.288 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 17 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.675.723 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 18 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.676.152 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 19 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.676.563 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 20 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.676.993 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 21 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.677.397 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 22 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.677.832 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 23 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.678.236 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 24 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.678.624 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 25 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.679.085 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 26 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.679.499 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 27 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.679.905 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 28 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.680.324 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 29 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.680.736 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 30 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.681.183 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 31 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.681.584 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 32 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.681.992 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 33 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.682.402 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 34 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.682.817 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 35 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.683.237 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 36 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.683.647 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 37 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.684.048 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 38 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.684.439 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 39 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.684.843 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 40 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.685.238 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 41 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.685.658 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 42 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.686.062 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 43 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.686.458 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 44 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.686.848 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 45 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.687.229 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 46 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.687.603 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 47 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.688.009 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 48 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.688.409 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 49 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.688.808 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 50 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.689.246 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 51 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.689.656 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 52 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.690.101 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 53 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.690.531 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 54 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.690.941 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 55 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.691.377 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 56 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.691.780 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 57 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.692.188 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 58 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.692.583 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 59 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.692.958 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 60 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.693.357 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 61 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.693.732 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 62 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.694.120 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 63 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.694.529 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 64 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.695.067 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 65 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.695.481 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 66 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.695.865 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 67 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.696.254 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 68 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.696.648 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 69 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.697.157 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 70 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.697.564 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 71 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.697.973 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 72 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.698.350 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 73 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.698.740 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 74 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.699.158 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 75 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.699.572 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 76 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.699.972 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 77 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.700.371 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 78 batch(es) to device, channel name: 14b2b416-af6a-11ee-a425-30fd658829ca [INFO] MD(164039,fffd15ffb0f0,python):2024-01-10-11:41:04.700.532 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:596] SendDataToAscend] ExecutionTree finished. Device queue sent number of batches: 78 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.887.163 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:978] CompileInner] Start compiling, phase: eval.1704858064728525312.281469945694832.0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.887.236 [mindspore/ccsrc/pipeline/jit/ps/event_message_print.cc:37] PrintEventMessage] Start compiling '_DataWrapper.construct' and it will take a while. Please wait... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.887.312 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1659] VmPipeline] This worker is initialized. No need to add worker action. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:04.887.350 [mindspore/ccsrc/backend/graph_compiler/transform.cc:573] CreateBackend] CreateBackend is: ge [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:04.887.371 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:04.887.386 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEBUG(164039,ffff87fd6440,python):2024-01-10-11:41:04.887.504 [mindspore/ccsrc/debug/debugger/debugger.cc:129] Init] Debugger got device_id: 0 [INFO] DEBUG(164039,ffff87fd6440,python):2024-01-10-11:41:04.887.522 [mindspore/ccsrc/debug/debugger/debugger.cc:131] Init] Debugger got device_target: Ascend [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.887.548 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1311] Run] Pipeline run [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.887.593 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start parse action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.889.657 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end parse action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.889.741 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start symbol_resolve action. [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.898.109 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380] Added global python symbol: {_check_is_tensor : Prim[_check_is_tensor]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.898.643 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_381{[0]: CNode_382, [1]: ValueNode logits, [2]: param_logits, [3]: CNode_383}, block: 0x4ee96260/mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/nn/loss/loss.py:777/ _check_is_tensor('logits', logits, self.cls_name)/ [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.899.172 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_384{[0]: CNode_382, [1]: ValueNode labels, [2]: param_labels, [3]: CNode_385}, block: 0x4ee96260/mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/nn/loss/loss.py:778/ _check_is_tensor('labels', labels, self.cls_name)/ [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.899.843 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_386{[0]: ValueNode Depend, [1]: CNode_387, [2]: CNode_388}, state: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_389{[0]: ValueNode MakeTuple, [1]: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_381{[0]: CNode_382, [1]: ValueNode logits, [2]: param_logits, [3]: CNode_383}, [2]: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_384{[0]: CNode_382, [1]: ValueNode labels, [2]: param_labels, [3]: CNode_385}} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.902.094 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_391, [1]: param_x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.902.380 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_392, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.902.654 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_393, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.902.927 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_394, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.903.190 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_395, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.903.464 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_396, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.903.729 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_397, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.903.991 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_398, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.904.246 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_399, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.904.505 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_400, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.904.763 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_401, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.905.022 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_402, [1]: x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.907.576 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_403] Added global python symbol: {len : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.907.741 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.908.089 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.908.259 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.908.746 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_403:CNode_405{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.908.879 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_404:x{[0]: CNode_406, [1]: param_фx, [2]: CNode_405} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.909.311 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_403:CNode_407{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.909.786 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_404] Added global python symbol: {len : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.909.856 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_403:CNode_408{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.909.977 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_403:x_shape{[0]: CNode_409, [1]: param_x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.910.111 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.910.409 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.910.686 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_404] Added global python symbol: {F : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.910.777 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_403] Added global python symbol: {F : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.910.832 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_403:CNode_410{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.911.150 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.915.573 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_411] Added global python symbol: {len : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.915.736 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.916.064 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.916.233 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.916.692 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_411:CNode_413{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.916.828 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_412:x{[0]: CNode_414, [1]: param_фx, [2]: CNode_413} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.917.255 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_411:CNode_415{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.917.721 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_412] Added global python symbol: {len : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.917.787 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_411:CNode_416{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.917.891 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_411:x_shape{[0]: CNode_417, [1]: param_x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.918.022 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.918.305 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.918.565 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_412] Added global python symbol: {F : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.918.654 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_411] Added global python symbol: {F : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.918.707 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_411:CNode_418{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.919.008 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.923.036 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_419] Added global python symbol: {len : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.923.197 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.923.529 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.923.695 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.924.146 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_419:CNode_421{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.924.278 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_420:x{[0]: CNode_422, [1]: param_фx, [2]: CNode_421} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.924.707 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_419:CNode_423{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.925.139 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_420] Added global python symbol: {len : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.925.204 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_419:CNode_424{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.925.308 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_419:x_shape{[0]: CNode_425, [1]: param_x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.925.439 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.925.734 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.925.997 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_420] Added global python symbol: {F : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.926.085 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_419] Added global python symbol: {F : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.926.139 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_419:CNode_426{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.926.441 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.928.530 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Flatten_construct_427] Added global python symbol: {F : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.931.427 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1261] ParseNameConstant] The NameConstant is bool:False [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.931.714 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:3 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.931.969 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.932.097 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1261] ParseNameConstant] The NameConstant is bool:True [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.932.834 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_pooling_MaxPool2d_construct_428] Added global python symbol: {isinstance : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.932.929 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_pooling_MaxPool2d_construct_429] Added global python symbol: {isinstance : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.932.987 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_pooling_MaxPool2d_construct_428 update var `isinstance` with node @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_430{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.pooling', [2]: ValueNode isinstance} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.933.140 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_pooling_MaxPool2d_construct_428] Added global python symbol: {tuple : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.933.220 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_pooling_MaxPool2d_construct_429] Added global python symbol: {tuple : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.933.270 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_pooling_MaxPool2d_construct_428 update var `tuple` with node @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_431{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.pooling', [2]: ValueNode tuple} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.933.603 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.933.731 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.933.933 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.934.037 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.934.317 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.945.959 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.946.119 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.946.699 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @canonicalize_axis_432:CNode_433{[0]: ValueNode check_axis_valid_434, [1]: param_axis, [2]: ndim}, block: 0x4efff6b0/canonicalize_axis_432, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1606/ check_axis_valid(axis, ndim)/ [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.946.855 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.947.152 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @canonicalize_axis_432:CNode_435{[0]: ValueNode Depend, [1]: CNode_436, [2]: CNode_437}, state: @canonicalize_axis_432:CNode_433{[0]: ValueNode check_axis_valid_434, [1]: @canonicalize_axis_432:param_axis, [2]: @canonicalize_axis_432:ndim{[0]: CNode_438}} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.947.456 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {isinstance : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.947.608 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {Tensor : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.948.187 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {int : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.948.652 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {bool : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.949.328 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {check_flatten_order_const : Prim[check_flatten_order]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.949.827 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @2↓flatten_440:CNode_441{[0]: CNode_442, [1]: param_order}, block: 0x4eca05f0/2↓flatten_440, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1615/ check_flatten_order_const(order)/ [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.950.236 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {rank_ : Prim[Rank]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.950.584 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.950.648 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.950.876 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {reshape_ : Prim[Reshape]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.951.061 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.951.356 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {ops : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.951.549 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.952.064 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {transpose_ : Prim[Transpose]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.952.479 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_443] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.952.582 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.952.645 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `shape_` with node @flatten_439:CNode_444{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode shape_} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.952.988 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_443] Added global python symbol: {rank_ : Prim[Rank]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.953.059 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `rank_` with node @flatten_439:CNode_445{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode rank_} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.953.351 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `start_dim` with node @flatten_439:param_start_dim [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.953.501 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.953.646 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `end_dim` with node @flatten_439:param_end_dim [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.953.768 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.954.039 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.954.091 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.954.316 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_443] Added global python symbol: {reshape_ : Prim[Reshape]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.954.386 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `reshape_` with node @flatten_439:CNode_446{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode reshape_} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.954.576 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.954.880 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_443] Added global python symbol: {flatten_ : Prim[Flatten]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.954.984 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {flatten_ : Prim[Flatten]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.955.049 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `flatten_` with node @flatten_439:CNode_447{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode flatten_} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.955.370 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `canonicalize_axis` with node ValueNode canonicalize_axis_432 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.955.807 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `check_dim_valid` with node ValueNode check_dim_valid_448 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.956.270 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @4↓flatten_449:CNode_450{[0]: ValueNode check_dim_valid_448, [1]: start_dim, [2]: end_dim}, block: 0x4f6229e0/4↓flatten_449, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1636/ check_dim_valid(start_dim, end_dim)/ [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.956.519 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.956.572 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.956.844 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.957.310 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.957.873 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.958.474 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.958.917 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @2↓flatten_440:CNode_451{[0]: ValueNode Depend, [1]: CNode_452, [2]: CNode_453}, state: @2↓flatten_440:CNode_441{[0]: @flatten_439:CNode_442{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode check_flatten_order_const}, [1]: @flatten_439:param_order} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.959.034 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @4↓flatten_449:CNode_454{[0]: ValueNode Depend, [1]: CNode_455, [2]: CNode_456}, state: @4↓flatten_449:CNode_450{[0]: ValueNode check_dim_valid_448, [1]: @4↓flatten_449:idx{[0]: ValueNode canonicalize_axis_432, [1]: param_start_dim, [2]: x_rank}, [2]: @4↓flatten_449:end_dim{[0]: ValueNode canonicalize_axis_432, [1]: param_end_dim, [2]: x_rank}} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.959.143 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.959.238 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.960.397 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:174] CheckFuncReturn] Function must has 'return' statement, but missing in function 'flatten' at /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1557. FuncGraph: ↓check_dim_valid_457. We will add a 'return None' statement automatically. [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.960.555 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:174] CheckFuncReturn] Function must has 'return' statement, but missing in function 'flatten' at /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1557. FuncGraph: ↓check_axis_valid_458. We will add a 'return None' statement automatically. [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.972.840 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [shape_459] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.980.613 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end symbol_resolve action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.980.664 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start graph_reusing action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.980.680 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.basic.DenseDense[True, None]_ID [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.980.704 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.conv.Conv2dConv2dodict_values([6, 16, 5, 1, 'valid', 0, False, ])_ID [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.980.716 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.conv.Conv2dConv2dodict_values([1, 6, 5, 1, 'valid', 0, False, ])_ID [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.980.730 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end graph_reusing action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.980.748 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start meta_unpack_prepare action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.981.509 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end meta_unpack_prepare action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.981.545 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pre_cconv action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.981.558 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pre_cconv action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:04.981.575 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start abstract_specialize action. [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:04.983.165 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_463{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:04.983.222 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:04.983.622 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_464{[0]: CNode_465}, new_node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_466{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:04.983.671 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_464{[0]: CNode_465}, new node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_464{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.984.733 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_equal_scalar_467] Added global python symbol: {F : } [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:04.985.091 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 4, Shape: NoShape), AbstractScalar(Type: Int64, Value: 3, Shape: NoShape)}, g: _equal_scalar_467 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:04.985.772 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_468:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_468:CNode_470{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:04.985.836 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_468:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_468:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:04.987.893 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_472{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:04.987.949 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:04.988.270 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_473{[0]: CNode_474}, new_node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_475{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:04.988.319 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_473{[0]: CNode_474}, new node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_473{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:04.988.957 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_476:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_476:CNode_477{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:04.989.020 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_476:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_476:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.994.833 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_logical_not_scala_478] Added global python symbol: {auto_generate : } [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:04.995.269 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Bool, Value: true, Shape: NoShape)}, g: _logical_not_scala_478 [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.996.883 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bool_not_479] Added global python symbol: {_get_cache_prim : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:04.997.016 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bool_not_479] Added global python symbol: {BoolNot : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.000.555 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_480] Added global python symbol: {str : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.000.992 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↻_get_cache_prim_for_pynative_481] Added global python symbol: {str : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.001.252 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↻_get_cache_prim_for_pynative_481 update var `str` with node @↵_get_cache_prim_for_pynative_482:param_фstr [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.001.467 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_480] Added global python symbol: {tuple : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.001.667 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] _get_cache_prim_for_pynative_483 update var `key` with node @_get_cache_prim_for_pynative_483:key{[0]: CNode_484, [1]: key, [2]: CNode_485} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.002.455 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_486] Added global python symbol: {str : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.003.035 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_486] Added global python symbol: {_PRIM_CACHE : {}} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.003.123 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_480] Added global python symbol: {_PRIM_CACHE : {}} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.003.367 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_486] Added global python symbol: {Primitive : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.003.454 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_480] Added global python symbol: {Primitive : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.004.127 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @✓↓_get_cache_prim_for_pynative_487:CNode_488{[0]: ValueNode MetaFuncGraph-unpack_call.489, [1]: CNode_490, [2]: param_фargs, [3]: param_фkwargs}, block: 0x4f907250/✓↓_get_cache_prim_for_pynative_487, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/_primitive_cache.py:84/ prim.__init__(*args, **kwargs)/ [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.004.706 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 2↓_get_cache_prim_for_pynative_491 update var `key` with node @↓_get_cache_prim_for_pynative_492:key{[0]: param_фstr, [1]: param_фkey} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.004.858 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @↻_get_cache_prim_for_pynative_493:CNode_494{[0]: ValueNode Depend, [1]: CNode_495, [2]: CNode_496}, state: @↻_get_cache_prim_for_pynative_493:CNode_497{[0]: ValueNode MetaFuncGraph-add.144, [1]: @↵_get_cache_prim_for_pynative_486:param_@CNode_497, [2]: ValueNode 1} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.004.965 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @✓↓_get_cache_prim_for_pynative_487:CNode_498{[0]: ValueNode Depend, [1]: CNode_499, [2]: CNode_500}, state: @✓↓_get_cache_prim_for_pynative_487:CNode_488{[0]: ValueNode MetaFuncGraph-unpack_call.489, [1]: @✓↓_get_cache_prim_for_pynative_487:CNode_490{[0]: ValueNode getattr, [1]: prim, [2]: ValueNode __init__}, [2]: @↵_get_cache_prim_for_pynative_486:param_фargs, [3]: @↵_get_cache_prim_for_pynative_486:param_фkwargs} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.006.305 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_501:CNode_502{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.006.371 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_501:CNode_503{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.006.412 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_501:CNode_504{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.007.016 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BoolNot. node: @bool_not_479:CNode_505{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x}, new_node: @bool_not_479:CNode_506{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.007.070 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BoolNot. node: @bool_not_479:CNode_505{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x}, new node: @bool_not_479:CNode_505{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.011.274 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_equal_string_507] Added global python symbol: {F : } [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.011.617 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: String, Value: C, Shape: NoShape), AbstractScalar(Type: String, Value: F, Shape: NoShape)}, g: _equal_string_507 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.013.102 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_508:CNode_509{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_508:CNode_510{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.013.166 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_508:CNode_509{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_508:CNode_509{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.014.414 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_neg_scalar_511] Added global python symbol: {F : } [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.014.745 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 1, Shape: NoShape)}, g: _neg_scalar_511 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.015.333 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarUsub. node: @_neg_scalar_512:CNode_513{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x}, new_node: @_neg_scalar_512:CNode_514{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.015.389 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarUsub. node: @_neg_scalar_512:CNode_513{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x}, new node: @_neg_scalar_512:CNode_513{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.016.042 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_515:CNode_516{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_515:CNode_517{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.016.105 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_515:CNode_516{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_515:CNode_516{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.016.530 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @3↓flatten_518:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput}, new_node: @3↓flatten_518:CNode_519{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.016.589 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @3↓flatten_518:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput}, new node: @3↓flatten_518:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.017.957 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_520] Added global python symbol: {F : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.018.620 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_520] Added global python symbol: {InSequence : } [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.018.972 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_520] Added global python symbol: {const_utils : } [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.019.461 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 4, Shape: NoShape), AbstractTuple{ element[0]: AbstractScalar(Type: Int64, Value: 0, Shape: NoShape), element[1]: AbstractScalar(Type: Int64, Value: 1, Shape: NoShape), sequence_nodes: {@✓3↓flatten_521:CNode_522{[0]: ValueNode MakeTuple, [1]: ValueNode 0, [2]: ValueNode 1}, elements_use_flags: {ptr: 0x4f8e2f40, value: [const vector]{0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: _number_in_tuple_520 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.022.789 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Flatten. node: @↓✓3↓flatten_523:CNode_524{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput}, new_node: @↓✓3↓flatten_523:CNode_525{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.022.855 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Flatten. node: @↓✓3↓flatten_523:CNode_524{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput}, new node: @↓✓3↓flatten_523:CNode_524{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.023.174 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_419:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_419:CNode_526{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.023.221 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_419:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_419:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.024.442 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_not_equal_scalar_527] Added global python symbol: {F : } [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.024.795 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 2, Shape: NoShape), AbstractScalar(Type: Int64, Value: 2, Shape: NoShape)}, g: _not_equal_scalar_527 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.025.490 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_528:CNode_529{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_528:CNode_530{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.025.553 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_528:CNode_529{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_528:CNode_529{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.027.520 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_532:CNode_533{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_532:CNode_534{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.027.595 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_532:CNode_533{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_532:CNode_533{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.028.766 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_535:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_535:CNode_536{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias, [3]: ValueNode 0} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.028.831 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_535:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_535:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias, [3]: ValueNode 0} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.029.097 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_537{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.029.143 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.029.316 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_411:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_411:CNode_538{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.029.360 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_411:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_411:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.030.235 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_539:CNode_540{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_539:CNode_541{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.030.300 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_539:CNode_540{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_539:CNode_540{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.032.075 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_542:CNode_543{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_542:CNode_544{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.032.138 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_542:CNode_543{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_542:CNode_543{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.033.291 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_545:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_545:CNode_546{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias, [3]: ValueNode 0} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.033.356 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_545:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_545:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias, [3]: ValueNode 0} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.033.611 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_547{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.033.659 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.033.852 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_403:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_403:CNode_548{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.033.897 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_403:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_403:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.034.721 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_549:CNode_550{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_549:CNode_551{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.034.784 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_549:CNode_550{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_549:CNode_550{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.036.569 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_552:CNode_553{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_552:CNode_554{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.036.634 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_552:CNode_553{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_552:CNode_553{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.037.811 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_555:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_555:CNode_556{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias, [3]: ValueNode 0} [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.037.875 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_555:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_555:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias, [3]: ValueNode 0} [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.038.761 [mindspore/ccsrc/pipeline/jit/ps/action.cc:282] AbstractAnalyze] function call depth: 0, simulate call depth: 0 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.038.888 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:222] Run] Specialize set top func graph context: {FuncGraph: mindspore_train_dataset_helper__DataWrapper_construct_460 Args: [0]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [1]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [2]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [3]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [4]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [5]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [6]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [7]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), Parent: } [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.046.183 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end abstract_specialize action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.046.237 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pack_expand action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.046.352 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pack_expand action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.046.384 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start auto_monad action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.047.371 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end auto_monad action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.047.415 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start inline action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.047.432 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end inline action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.047.450 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pre_auto_parallel action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.047.476 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pre_auto_parallel action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.047.493 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pipeline_split action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.047.508 [mindspore/ccsrc/pipeline/jit/ps/pipeline_split.cc:247] PipelineSplit] Only auto_parallel and semi_auto_parallel support pipeline split. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.047.523 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pipeline_split action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.047.539 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start optimize action. [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.050.411 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [fill_557] Added global python symbol: {cast_ : Prim[Cast]} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.050.634 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] fill_557 update var `value` with node @fill_557:value{[0]: CNode_558, [1]: param_value, [2]: param_type} [INFO] PARSER(164039,ffff87fd6440,python):2024-01-10-11:41:05.050.883 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [fill_557] Added global python symbol: {fillv2_ : Prim[FillV2]} [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.056.615 [mindspore/ccsrc/frontend/parallel/pynative_shard/pynative_shard.cc:334] PynativeShard] Only auto_parallel and semi_auto_parallel support pynative shard [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.056.675 [mindspore/ccsrc/frontend/parallel/step_parallel.cc:3009] StepParallel] Strategies would be ignored in data_parallel, shard() only valid in [semi_]auto_parallel. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.059.195 [mindspore/ccsrc/pipeline/jit/ps/action.cc:282] AbstractAnalyze] function call depth: 0, simulate call depth: 0 [INFO] ANALYZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.059.986 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:222] Run] Specialize set top func graph context: {FuncGraph: 382_mindspore_train_dataset_helper__DataWrapper_construct_559 Args: [0]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [1]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [2]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [3]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [4]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [5]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [6]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), [7]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2d1b22b0, value: ValueAny), value: ValueAny), Parent: } [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.062.482 [mindspore/ccsrc/frontend/parallel/pynative_shard/pynative_shard.cc:334] PynativeShard] Only auto_parallel and semi_auto_parallel support pynative shard [INFO] OPTIMIZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.064.645 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] OPTIMIZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.065.071 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] OPTIMIZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.065.393 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.065.467 [mindspore/ccsrc/frontend/parallel/cache_embedding/cache_embedding.cc:702] AddCacheEmbedding] Parameters are all not cache enable. [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.065.908 [mindspore/ccsrc/frontend/parallel/pass/assign_add_opt.cc:120] AssignAddOpt] parallel::g_device_manager is not initialized. [INFO] OPTIMIZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.065.970 [mindspore/ccsrc/frontend/optimizer/comm_op_reuse_tag.cc:59] AddCommOpReuseTag] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.065.991 [mindspore/ccsrc/frontend/parallel/pass/overlap_opt_shard_in_pipeline.cc:70] OverlapOptShardInPipeline] parallel::g_device_manager is not initialized. [INFO] OPTIMIZER(164039,ffff87fd6440,python):2024-01-10-11:41:05.066.008 [mindspore/ccsrc/frontend/optimizer/grouped_pairwise_exchange_alltoall.cc:673] SetGroupedPairwiseExchangeAllToAll] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.066.028 [mindspore/ccsrc/frontend/parallel/pass/overlap_gradmatmul_and_gradallreduce.cc:358] OverlapGradMatmulAndGradAllreduce] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.066.055 [mindspore/ccsrc/frontend/parallel/pass/split_matmul_comm_elementwise_fp.cc:184] SplitMatmulCommElementwiseFp] SplitMatmulCommElementwiseFp is only support under [semi_]auto_parallel, skip it. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.066.083 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end optimize action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.066.102 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start auto_monad_reorder action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.066.229 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end auto_monad_reorder action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.066.254 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start get_jit_bprop_graph action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.066.266 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end get_jit_bprop_graph action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.066.281 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start eliminate_special_op_node action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.066.861 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end eliminate_special_op_node action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.066.903 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start validate action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.067.015 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end validate action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.067.039 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start distribtued_split action. [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.067.058 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:372] GenerateStrategy] Current parallel mode is data_parallel [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.067.070 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:384] GenerateStrategy] Generated distributed strategy is 1 [INFO] PARALLEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.067.189 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:1270] Run] All nodes are on this precoess so there's no need to build and split distributed graph. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.067.208 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end distribtued_split action. [INFO] PROFILER(164039,ffff87fd6440,python):2024-01-10-11:41:05.067.239 [mindspore/ccsrc/plugin/device/ascend/hal/profiler/parallel_strategy_profiling.cc:48] IsProfilingParallelStrategyEnabled] Profiling parallel strategy is disabled. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.067.258 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start task_emit action. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.067.400 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.067.421 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.067.435 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1252] SetRunMode] Run graph mode with kernel by kernel by configuration. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.067.480 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:546] CompileGraphs] Status record: start compile function graph: 458_382_mindspore_train_dataset_helper__DataWrapper_construct_560 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.067.627 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unify_mindir_pm_0_erase_invalid_micro_depend in 2.52 us [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.035 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.059 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.190 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.207 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.232 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.245 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.263 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.278 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.293 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.305 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.319 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.331 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.346 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.356 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.370 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.383 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.397 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.410 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.424 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.434 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.450 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.461 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.484 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.498 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.548 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.564 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.580 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.595 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.609 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.621 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.636 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.647 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.662 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.674 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.689 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.703 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.721 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.735 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.749 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.760 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.774 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.785 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.803 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.814 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.831 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.843 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.865 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.877 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.895 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.906 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.924 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.935 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.951 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.962 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.977 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.068.988 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.069.005 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.069.018 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.069.037 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.069.048 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.069.063 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.069.074 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.069.092 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.069.104 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.069.121 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.069.131 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.069.178 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:727] CompileGraph] Compile graph: 458_382_mindspore_train_dataset_helper__DataWrapper_construct_560, Split segments size: 2 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.069.218 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:762] CompileGraphFromSegment] Compile normal segment, the first node: @458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:CNode_561{[0]: ValueNode Load, [1]: param_fc3.bias, [2]: ValueNode U} [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.069.420 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:421] CompileGraph] Status record: start compile graph. [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.069.457 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:1940] ConstructKernelGraph] Create graph: 3 [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.070.117 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:2697] ConstructOutput] Output:@458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:562{[0]: ValueNode Depend, [1]: 562, [2]: CNode_563} [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.071.110 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:193] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:3 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.071.618 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 38.47 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.071.914 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 252.32 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.072.333 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:154] CommonUnifyMindIR] start common unify mindir opt graph:3 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.072.743 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.072.954 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 207.95 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.072.980 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.073.016 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 33.12 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.073.036 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.073.051 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 14.8 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.073.066 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.073.271 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 197.5 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.073.732 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:79] BackendCommonOptimization] Status record: start common optimization. graph id: 3 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.074.148 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: add_dynamic_shape_attr [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.074.454 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_0_add_dynamic_shape_attr in 301.6 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.074.485 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.074.551 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_1_convert_dynamic_broadcast_to in 61.02 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.074.784 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_2_convert_const_input_to_attr in 204.81 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.074.977 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_3_custom_op_const_input_to_attr in 163.63 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.075.146 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 141.96 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.075.635 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 457.16 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.075.671 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 8.8 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.075.829 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_7_flatten_concat_fission in 133.45 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.075.980 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 123.11 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.076.125 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_9_broadcast_to_fusion in 119.44 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.076.319 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 166.72 us [WARNING] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.076.736 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:92] BackendCommonOptimization] [PROF]backend_common_optimization costs 3006 usec. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.076.764 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:99] BackendCommonOptimization] Status record: end common optimization. graph id: 3 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.077.419 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_0_renorm_split in 152.5 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.077.450 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.077.824 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_1_reduce_axis_update in 363.74 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.077.851 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim HistogramFixedWidth not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.077.868 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_2_histogram_fixed_width_fusion in 16.94 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.077.883 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim ClipByNorm not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.077.898 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_3_clip_by_norm_fission in 14.41 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.077.913 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.077.937 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_4_tensor_arry_add_flow_cond1 in 22.97 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.077.951 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayGather not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.077.964 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_5_tensor_arry_add_flow_cond2 in 11.93 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.077.977 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.077.990 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_6_ge_tensor_arry_cast_index in 11.59 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.078.003 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArray not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.078.017 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_7_tensor_array_prepare in 12.38 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.078.029 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.078.068 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_8_space_to_batch_nd_attr_update in 34.65 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.078.085 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.078.114 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_9_batch_to_space_nd_attr_update in 26.66 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.078.134 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: avg_pool_grad_unify_mindir [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.078.170 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_10_avg_pool_grad_unify_mindir in 33.74 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.078.334 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_11_adam_weight_decay_unify_mindir in 140.23 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.078.489 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_12_cdist_fission in 128.64 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.078.637 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_13_cdist_grad_fission in 122.78 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.078.811 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_14_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 148.34 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.078.995 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_15_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 155.9 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.079.735 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_16_sparse_softmax_cross_entropy_with_logits_unify_mindir in 699.19 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.079.939 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_17_dropout_unify_mindir1 in 159.82 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.079.986 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.158 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_18_dropoutgrad_unify_mindir in 167.02 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.180 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.197 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_19_neighbor_exchange_unify_mindir in 16.02 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.212 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.226 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_20_neighbor_exchange_v2_unify_mindir in 13.11 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.241 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.255 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_21_neighbor_exchange_v2_grad_unify_mindir in 12.58 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.270 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.284 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_22_all_to_all_unify_mindir in 12.24 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.300 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim QuantDTypeCast not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.314 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_23_quant_dtype_cst_adjust in 15.19 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.330 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim FSEDecode not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.344 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_24_fse_decode_adjust in 14.57 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.358 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.371 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_25_bn_split in 12.44 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.384 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.445 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_26_bn_grad_unify_mindir in 57.42 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.463 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.478 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_27_bn_grad_split in 13.76 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.500 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.519 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_28_batchnorm_to_bninfer in 17.61 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.532 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.548 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_29_batchnormgrad_to_bninfergrad in 14.04 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.561 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.575 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_30_batch_norm_grad_infer_fission in 12.09 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.876 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_31_ascend_mindir_op_adapter in 277.66 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.080.905 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_32_DropoutGenMask in 1.25 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.081.091 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_33_lamb_fission_ge in 164.9 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.081.459 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_34_print_insert_placeholder_for_tensor_name in 338.67 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.081.674 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_35_getnext_for_ge in 183.8 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.081.878 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_36_sync_bn_split in 150.94 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.082.062 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_37_sync_bn_grad_split in 153.62 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.082.236 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_38_adaptive_max_pool2d_ge_fusion in 145.61 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.082.413 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_39_avg_pool_grad_for_ge in 149.74 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.082.437 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:260] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.082.853 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_40_FlashAttentionFusionV1 in 410.03 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.082.885 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:374] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.083.294 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_41_FlashAttentionFusionV1 in 406.98 us [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.084.629 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164039,ffff87fd6440,python):2024-01-10-11:41:05.084.671 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:05.084.686 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:132] GetRunMode] RunMode::kKernelMode [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.085.216 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 483.28 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.085.992 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_0_seed_adapter in 501.02 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.086.026 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: env_op_attr_update [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.086.307 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_1_env_op_attr_update in 271.49 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.086.493 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_2_process call inline in 154.51 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.086.592 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_3_expander_fallback in 66.6 us [WARNING] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.086.830 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:139] GEBackendOptimizeACL] [PROF]ascend_backend_optimize_acl costs 1358 usec. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.086.878 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] GetNext is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.086.976 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2D is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.087.177 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPool is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.087.251 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2D is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.087.382 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPool is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.087.502 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.087.725 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.087.916 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.088.292 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] SoftmaxCrossEntropyWithLogits is not defined in opdef. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.088.745 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 59.26 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.089.165 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 377.3 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.089.590 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 378.84 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.090.204 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 575.42 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.090.727 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_4_insert_type_transform_op in 480.35 us [WARNING] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.090.975 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:179] GEBackendOptimizeACLAfterKernelSelect] [PROF]ascend_backend_optimize_acl_after_kernel_select costs 2338 usec. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.091.065 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_0_DropoutGenMask in 0.83 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.091.238 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_1_cse in 137.44 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.092.089 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_redundant_op in 814.94 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.092.258 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_3_eliminate_maketuple_getitem in 134.41 us [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.092.288 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_4_MergeTransData in 1.42 us [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.092.687 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive GetNext [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:41:05.092.883 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:467] ConvertAny] Value: ValueTuple [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.092.976 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive GetNext [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.093.002 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2D [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:41:05.093.080 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.093.175 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2D [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.093.199 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.093.275 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.093.296 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPool [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.093.388 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPool [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.093.413 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2D [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:41:05.093.481 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.093.561 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2D [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.093.583 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.093.651 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.093.671 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPool [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.093.767 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPool [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.093.790 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Flatten [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.093.864 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Flatten [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.093.888 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.093.979 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.094.003 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.094.120 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.094.147 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.094.213 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.094.237 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.094.315 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.094.338 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.094.426 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.094.450 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.094.524 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.094.545 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.094.623 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.094.646 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.094.733 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.094.757 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.094.837 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.094.857 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.094.938 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.094.961 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive OneHot [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.095.101 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive OneHot [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.095.125 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive SoftmaxCrossEntropyWithLogits [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.095.231 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive SoftmaxCrossEntropyWithLogits [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.095.254 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReduceMean [INFO] KERNEL(164039,ffff87fd6440,python):2024-01-10-11:41:05.095.363 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReduceMean [WARNING] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:05.095.384 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:262] CreateKernel] [PROF]create_kernel costs 2718 usec. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.095.525 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1_6, index: 0 to input Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2_6, index: 0 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.095.560 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/Reshape-op0_6, index: 0 to input Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6, index: 0 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.095.600 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/Reshape-op1_6, index: 0 to input Default/GetNext-op1_6, index: 1 [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.095.617 [mindspore/ccsrc/backend/common/session/session_basic.cc:1140] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 3 start [INFO] DEBUG(164039,ffff87fd6440,python):2024-01-10-11:41:05.095.630 [mindspore/ccsrc/debug/summary/summary.cc:33] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 3 start [INFO] DEBUG(164039,ffff87fd6440,python):2024-01-10-11:41:05.095.646 [mindspore/ccsrc/debug/summary/summary.cc:54] RecurseSetSummaryNodesForAllGraphs] The total summary nodes is: 0 for graph: 3 [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.095.872 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:891] PrintGraphExecuteOrder] Graph 3 execution order: [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.095.920 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[0], node name[Default/GetNext-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:outputs{[0]: ValueNode GetNext}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.095.971 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[1], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv1-Conv2d/Conv2D-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode Conv2D, [1]: 562, [2]: CNode_564}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.096.011 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[2], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op4_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReLU, [1]: 562}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.096.050 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[3], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op3_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode MaxPool, [1]: 562}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.096.092 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[4], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/Conv2D-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode Conv2D, [1]: 562, [2]: CNode_565}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.096.127 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[5], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op5_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReLU, [1]: 562}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.096.162 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[6], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode MaxPool, [1]: 562}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.096.203 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[7], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_Flatten, [1]: 562}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.096.256 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[8], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/MatMul-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode MatMul, [1]: 562, [2]: CNode_566}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.096.302 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[9], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/BiasAdd-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_BiasAdd, [1]: 562, [2]: CNode_567, [3]: ValueNode 0}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.096.339 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[10], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op6_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReLU, [1]: 562}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.096.379 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[11], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/MatMul-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode MatMul, [1]: 562, [2]: CNode_568}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.096.421 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[12], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/BiasAdd-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_BiasAdd, [1]: 562, [2]: CNode_569, [3]: ValueNode 0}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.096.454 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[13], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op7_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReLU, [1]: 562}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.096.493 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[14], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/MatMul-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode MatMul, [1]: 562, [2]: CNode_570}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.096.530 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[15], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_BiasAdd, [1]: 562, [2]: CNode_561, [3]: ValueNode 0}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.096.581 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[16], node name[Default/Reshape-op0_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_Reshape, [1]: 562, [2]: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10])}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.096.626 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[17], node name[Default/Reshape-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_Reshape, [1]: 562, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[32])}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.096.700 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[18], node name[Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/OneHot-op0_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_OneHot, [1]: 562, [2]: ValueNode Tensor(shape=[], dtype=Int64, value=10), [3]: ValueNode Tensor(shape=[], dtype=Float32, value=1), [4]: ValueNode Tensor(shape=[], dtype=Float32, value=0), [5]: ValueNode -1}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.096.747 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[19], node name[Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/SoftmaxCrossEntropyWithLogits-op0_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode SoftmaxCrossEntropyWithLogits, [1]: 562, [2]: 562}] [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.096.793 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[20], node name[Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/ReduceMean-op0_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReduceMean, [1]: 562, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), [3]: ValueNode false}] [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.096.930 [mindspore/ccsrc/backend/common/somas/somas.cc:143] Assign] Start Somas Assign for graph 3 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.097.047 [mindspore/ccsrc/backend/common/somas/somas.cc:151] Assign] Somas Configure success, configuration info: Device Name: Ascend Run by execution order: 1 Enable debug log: 0 Debug log path: . [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.097.061 [mindspore/ccsrc/backend/common/somas/somas.cc:154] Assign] Start Initialize SOMAS Model [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.097.540 [mindspore/ccsrc/backend/common/somas/somas.cc:1065] GraphOutputProcess] Set 3 graph output tensors' aligned size to 0. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.097.747 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 7 output 8 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.097.777 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 16 output 17 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.097.793 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 1 output 18 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.097.807 [mindspore/ccsrc/backend/common/somas/somas.cc:1135] RefNodeProcess] Special Tensor total size: RefNode: input 53760 output 52608 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.097.836 [mindspore/ccsrc/backend/common/somas/somas.cc:555] InitSomasModel] Created 1 streams (0 groups), 21 nodes, 23 tensors, 3 union tensors lists, and 0 contiguous tensors lists [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.010 [mindspore/ccsrc/backend/common/somas/somas.cc:157] Assign] End Initialize SOMAS Model [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.026 [mindspore/ccsrc/backend/common/somas/somas.cc:176] Assign] Start Computing Conflict Matrix [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.038 [mindspore/ccsrc/backend/common/somas/somas.cc:1286] ComputeBasicMatrix] Start Conflict Computing (Bitset Model) [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.054 [mindspore/ccsrc/backend/common/somas/somas.cc:1291] ComputeBasicMatrix] Start Bitset [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.081 [mindspore/ccsrc/backend/common/somas/somas.cc:1299] ComputeBasicMatrix] Start Path Computing [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.094 [mindspore/ccsrc/backend/common/somas/somas.cc:1307] ComputeBasicMatrix] End Path Computing [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.104 [mindspore/ccsrc/backend/common/somas/somas.cc:1309] ComputeBasicMatrix] Start Tensor Relation Computing [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.149 [mindspore/ccsrc/backend/common/somas/somas.cc:1462] ComputeMultiTensorConflicts] Start Computing Conflicts Pairs, tensors list size is 23 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.177 [mindspore/ccsrc/backend/common/somas/somas.cc:1469] ComputeMultiTensorConflicts] End Computing Conflicts Pairs (time taken 0ms) [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.188 [mindspore/ccsrc/backend/common/somas/somas.cc:1367] ComputeBasicMatrix] End Basic Conflict Computing (Bitset Model)(time taken 0ms) [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.211 [mindspore/ccsrc/backend/common/somas/somas.cc:178] Assign] End Computing Conflict Matrix [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.222 [mindspore/ccsrc/backend/common/somas/somas.cc:1533] Solve] Somas Assign start... [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.243 [mindspore/ccsrc/backend/common/somas/somas.cc:1555] Solve] Start Solving [INFO] PRE_ACT(164039,fffe9dffb0f0,python):2024-01-10-11:41:05.098.380 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 20 tensors [INFO] PRE_ACT(164039,fffe9cff90f0,python):2024-01-10-11:41:05.098.387 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 20 tensors [INFO] PRE_ACT(164039,fffe9e7fc0f0,python):2024-01-10-11:41:05.098.372 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 20 tensors [INFO] PRE_ACT(164039,fffe9d7fa0f0,python):2024-01-10-11:41:05.098.400 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 20 tensors [INFO] PRE_ACT(164039,fffe9cff90f0,python):2024-01-10-11:41:05.098.487 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 1/4 1205760 Bytes (0.00112295 GB) Shared Objects size(>), index(<) bestfit [INFO] PRE_ACT(164039,fffe9dffb0f0,python):2024-01-10-11:41:05.098.468 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 2/4 1205760 Bytes (0.00112295 GB) Shared Objects size(>), index(<) smallest [INFO] PRE_ACT(164039,fffe9e7fc0f0,python):2024-01-10-11:41:05.098.493 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 3/4 1205760 Bytes (0.00112295 GB) Single Object size(>), index(<) bestfit [INFO] PRE_ACT(164039,fffe9d7fa0f0,python):2024-01-10-11:41:05.098.530 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 4/4 1205760 Bytes (0.00112295 GB) Single Object size(>), index(<) smallest [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.553 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:176] Solving] SOMAS SOLVER RESUME: [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.568 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:177] Solving] Best Solution:[1/4] [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.585 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:178] Solving] Best result:1205760 Bytes 0.00112295 GB (0.00112295 GB + 0 GB from lifelong tensors) [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.596 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:181] Solving] Best timing:0 ms [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.607 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:182] Solving] Best algorithm: Shared Objects [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.624 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:183] Solving] Best sorting strategy: size(>), index(<) [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.635 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:184] Solving] Best offset strategy: bestfit [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.646 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:185] Solving] Time elapsed: 0 ms [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.657 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:186] Solving] Spread:0 %% [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.702 [mindspore/ccsrc/backend/common/somas/somas.cc:1564] Solve] End Solving [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.733 [mindspore/ccsrc/backend/common/somas/somas.cc:2096] GenGraphStatisticInfo] Lower Bound: 1205760 (0.00112295 GB), Upper Bound: 2039296 (0.00189924 GB) [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.746 [mindspore/ccsrc/backend/common/somas/somas.cc:2099] GenGraphStatisticInfo] Total Dynamic Size (Upper Bound): 2039296 Theoretical Optimal Size (Lower Bound): 1205760 Total Workspace Size: 0 Total Communication Input Tensor Size: 0 Total Communication Output Tensor Size: 0 Total LifeLong All Tensor Size: 0 Total LifeLong Start Tensor Size: 0 Total LifeLong End Tensor Size: 2560 Reused Size(Allocate Size): 0 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.758 [mindspore/ccsrc/backend/common/somas/somas.cc:1583] Solve] Somas Assign end. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.805 [mindspore/ccsrc/backend/common/somas/somas.cc:380] UpdateSomasResultToGraph] Merged Block size: 3 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.821 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 0, offset: 602624, size: 602624 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.832 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 1, offset: 0, size: 602624 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.843 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 2, offset: 1205248, size: 512 [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.854 [mindspore/ccsrc/backend/common/somas/somas.cc:189] Assign] Somas Allocate end. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.865 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:310] DoSomas] Somas allocate success for graph 3 somas size: 1205760 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.098.963 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:801] CreateDeviceAddress] Status record: start create device address. graph id: 3 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:05.099.116 [mindspore/ccsrc/runtime/device/device_address_utils.cc:454] CreateValueNodeDeviceAddress] No device address for value node:Default/data-9_6, debug name:ValueNode U [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:05.100.078 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6, index is 0; cur kernel is Default/Reshape-op0_6, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:05.100.123 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6, index is 0; cur kernel is Default/Reshape-op0_6, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:05.100.151 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/GetNext-op1_6, index is 1; cur kernel is Default/Reshape-op1_6, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:05.100.190 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/GetNext-op1_6, index is 1; cur kernel is Default/Reshape-op1_6, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:05.100.213 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2_6, index is 0; cur kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1_6, index is 0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:05.100.236 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2_6, index is 0; cur kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1_6, index is 0 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.100.254 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:808] CreateDeviceAddress] Status record: end create device address. graph id: 3 [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.100.341 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1129] CacheGraphOutputToFrontNodeWithIndex] Get graph backend output nodes. [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.100.388 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1137] CacheGraphOutputToFrontNodeWithIndex] Get graph front output nodes. [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.100.429 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1155] CacheGraphOutputToFrontNodeWithIndex] Backend output: Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/ReduceMean-op0_6 with index: 0 map to front node: Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits-op0_6 with index: 0 [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.100.444 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1155] CacheGraphOutputToFrontNodeWithIndex] Backend output: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6 with index: 0 map to front node: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op0_6 with index: 0 [INFO] SESSION(164039,ffff87fd6440,python):2024-01-10-11:41:05.100.457 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1155] CacheGraphOutputToFrontNodeWithIndex] Backend output: Default/GetNext-op1_6 with index: 1 map to front node: Default/GetNext-op0_6 with index: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.100.496 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:507] CompileGraph] Status record: end compile graph. graph id: 3 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.100.768 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:805] CompileGraphFromSegment] Compile cut segment, the cut node: @458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:CNode_571{[0]: ValueNode Return, [1]: 562} [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.100.950 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:521] Transform] Graph(kernel_graph_3) transforms actor begin, strategy:pipeline_with_execution_order [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.100.987 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2619] PersistDeviceTensorForValueNode] The device address is not exist: ValueNode_572(U) [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.101.115 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1101] BuildDataSourceActor] Create queue data source actor: kernel_graph_3_DeviceDSActor_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.101.398 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1232] BuildLoopCountActor] Create loop count actor: kernel_graph_3_LoopCountActor [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.101.428 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1265] BuildOutputActor] Create output actor: kernel_graph_3_OutputActor [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.101.449 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1285] BuildDataPrepareActor] Create data prepare actor: kernel_graph_3_DataPrepareActor [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.101.516 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:866] CacheGraphOutputToActor] Cache graph 3 output node:Default/GetNext-op1_6 debug string:@kernel_graph_3:outputs{[0]: ValueNode GetNext} with index:1 to actor:kernel_graph_3_DeviceDSActor_3, from front node:Default/GetNext-op0_6 debug string:@458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:outputs{[0]: ValueNode GetNext} with index:1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.101.589 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:866] CacheGraphOutputToActor] Cache graph 3 output node:Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6 debug string:@kernel_graph_3:562{[0]: ValueNode PrimFunc_BiasAdd, [1]: 562, [2]: CNode_561, [3]: ValueNode 0} with index:0 to actor:Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6, from front node:Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op0_6 debug string:@458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:562{[0]: ValueNode PrimFunc_BiasAdd, [1]: 562, [2]: CNode_561, [3]: ValueNode 0} with index:0 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.101.609 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:809] AddSomasInfoForGraphOutput] The graph 3 output node:Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6 with index: 0 somas enable or not: 1, somas offset: 545280, aligned size: 1536 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.101.710 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:866] CacheGraphOutputToActor] Cache graph 3 output node:Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/ReduceMean-op0_6 debug string:@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReduceMean, [1]: 562, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), [3]: ValueNode false} with index:0 to actor:Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/ReduceMean-op0_6, from front node:Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits-op0_6 debug string:@458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:562{[0]: ValueNode SparseSoftmaxCrossEntropyWithLogits, [1]: 562, [2]: 562} with index:0 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.101.737 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:809] AddSomasInfoForGraphOutput] The graph 3 output node:Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/ReduceMean-op0_6 with index: 0 somas enable or not: 1, somas offset: 546816, aligned size: 512 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.101.756 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1518] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_3 start. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.101.814 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1520] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_3 end. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.102.509 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 1_actor_set_kernel_graph_3_memory_actor_insert in 19.71 us [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.102.543 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 2_actor_set_kernel_graph_3_invalid_data_arrow_elimination in 2.6 us [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.102.637 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 3_actor_set_kernel_graph_3_multi_actor_fusion in 71.55 us [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.102.666 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 4_actor_set_kernel_graph_3_batch_data_arrow_fusion in 8.59 us [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.102.688 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:554] Transform] Graph(kernel_graph_3) transforms actor end. [WARNING] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.103.129 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:612] CompileGraphs] [PROF]compile_func_graph costs 35613 usec. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.103.172 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:615] CompileGraphs] Status record: end compile function graph: 458_382_mindspore_train_dataset_helper__DataWrapper_construct_560, produce actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.103.195 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end task_emit action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.103.216 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:268] SetLoopCount] Change vm_loop_flag to 0, set loop_size to 1 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.103.233 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start execute action. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.103.251 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end execute action. TotalTime = 0.215679, [19] [parse]: 0.00209537 [symbol_resolve]: 0.0909061, [1] [Cycle 1]: 0.0903026, [1] [resolve]: 0.0902795 [graph_reusing]: 7.898e-05 [meta_unpack_prepare]: 0.00078323 [pre_cconv]: 2.598e-05 [abstract_specialize]: 0.0646401 [pack_expand]: 0.00013681 [auto_monad]: 0.00101415 [inline]: 3.164e-05 [pre_auto_parallel]: 3.696e-05 [pipeline_split]: 4.044e-05 [optimize]: 0.0185543, [35] [py_interpret_to_execute]: 0.00022509 [rewriter_before_opt_a]: 0.00110811 [opt_a]: 0.0141799, [2] [Cycle 1]: 0.0100812, [30] [expand_dump_flag]: 1.761e-05 [switch_simplify]: 0.00033036 [a_1]: 0.0032082 [recompute_prepare]: 3.356e-05 [updatestate_depend_eliminate]: 0.00033128 [updatestate_assign_eliminate]: 4.721e-05 [updatestate_loads_eliminate]: 0.00021846 [parameter_eliminate]: 3.77e-06 [a_2]: 0.00060759 [accelerated_algorithm]: 2.937e-05 [pynative_shard]: 4.229e-05 [auto_parallel]: 4.51e-06 [parallel]: 3.337e-05 [merge_comm]: 1.673e-05 [allreduce_fusion]: 8.61e-06 [virtual_dataset]: 1.901e-05 [get_grad_eliminate_]: 1.563e-05 [virtual_output]: 1.486e-05 [merge_forward]: 2.811e-05 [cell_reuse_recompute_pass]: 9.79999e-07 [cell_reuse_handle_not_recompute_node_pass]: 4.566e-05 [meta_fg_expand]: 2.744e-05 [after_resolve]: 2.333e-05 [a_after_grad]: 2.014e-05 [renormalize]: 0.00427477 [real_op_eliminate]: 2.554e-05 [auto_monad_grad]: 5.44e-06 [auto_monad_eliminator]: 0.00011913 [cse]: 0.00016019 [a_3]: 0.00015338 [Cycle 2]: 0.0013242, [30] [expand_dump_flag]: 1.68e-06 [switch_simplify]: 1.652e-05 [a_1]: 0.00023763 [recompute_prepare]: 1.421e-05 [updatestate_depend_eliminate]: 2.907e-05 [updatestate_assign_eliminate]: 2.516e-05 [updatestate_loads_eliminate]: 2.497e-05 [parameter_eliminate]: 1.88e-06 [a_2]: 0.00028326 [accelerated_algorithm]: 2.682e-05 [pynative_shard]: 4.088e-05 [auto_parallel]: 4.52e-06 [parallel]: 6.35e-06 [merge_comm]: 1.032e-05 [allreduce_fusion]: 6.04e-06 [virtual_dataset]: 1.773e-05 [get_grad_eliminate_]: 1.513e-05 [virtual_output]: 1.513e-05 [merge_forward]: 2.13e-05 [cell_reuse_recompute_pass]: 5.59994e-07 [cell_reuse_handle_not_recompute_node_pass]: 4.095e-05 [meta_fg_expand]: 1.717e-05 [after_resolve]: 2.211e-05 [a_after_grad]: 1.942e-05 [renormalize]: 6.99947e-08 [real_op_eliminate]: 1.515e-05 [auto_monad_grad]: 2.38e-06 [auto_monad_eliminator]: 6.043e-05 [cse]: 9.277e-05 [a_3]: 0.00012417 [py_interpret_to_execute_after_opt_a]: 3.288e-05 [slice_cell_reuse_recomputed_activation]: 2.65001e-06 [rewriter_after_opt_a]: 0.00089051 [convert_after_rewriter]: 3.346e-05 [order_py_execute_after_rewriter]: 2.098e-05 [opt_b]: 0.00067923, [1] [Cycle 1]: 0.00067364, [7] [b_1]: 0.00042011 [b_2]: 1.822e-05 [updatestate_depend_eliminate]: 2.284e-05 [updatestate_assign_eliminate]: 2.12e-05 [updatestate_loads_eliminate]: 2.392e-05 [renormalize]: 3.453e-05 [cse]: 9.158e-05 [cconv]: 3.474e-05 [opt_after_cconv]: 0.00028416, [1] [Cycle 1]: 0.00027883, [7] [c_1]: 5.917e-05 [parameter_eliminate]: 1.45e-06 [updatestate_depend_eliminate]: 2.447e-05 [updatestate_assign_eliminate]: 2.415e-05 [updatestate_loads_eliminate]: 2.365e-05 [cse]: 7.916e-05 [renormalize]: 3.228e-05 [remove_dup_value]: 9.968e-05 [tuple_transform]: 0.00021291, [1] [Cycle 1]: 0.00020803, [3] [d_1]: 0.00011836 [d_2]: 4.261e-05 [renormalize]: 2.982e-05 [add_cache_embedding]: 6.05e-05 [add_recomputation]: 0.00026682 [cse_after_recomputation]: 9.166e-05, [1] [Cycle 1]: 8.608e-05, [1] [cse]: 7.994e-05 [environ_conv]: 2.649e-05 [label_micro_interleaved_index]: 2.34e-06 [label_fine_grained_interleaved_index]: 2.25e-06 [assign_add_opt]: 3.168e-05 [slice_recompute_activation]: 2.33e-06 [micro_interleaved_order_control]: 1.81e-06 [full_micro_interleaved_order_control]: 1.73e-06 [comp_comm_scheduling]: 2.14e-06 [reorder_send_recv_between_fp_bp]: 2.02e-06 [comm_op_add_attrs]: 1.05e-06 [add_comm_op_reuse_tag]: 2.039e-05 [overlap_opt_shard_in_pipeline]: 1.285e-05 [grouped_pairwise_exchange_alltoall]: 1.304e-05 [overlap_recompute_and_grad_model_parallel]: 1.66e-06 [overlap_grad_matmul_and_grad_allreduce]: 2.083e-05 [split_matmul_comm_elemetwise]: 1.443e-05 [split_layernorm_comm]: 1.84e-06 [process_send_recv_for_ge]: 8.49999e-07 [handle_group_info]: 1.03e-06 [auto_monad_reorder]: 0.00014301 [get_jit_bprop_graph]: 2.308e-05 [eliminate_special_op_node]: 0.00060603 [validate]: 0.00012872 [distribtued_split]: 0.00018028 [task_emit]: 0.0359493 [execute]: 2.959e-05 Sums parse : 0.002095s : 0.99% symbol_resolve.resolve : 0.090280s : 42.71% graph_reusing : 0.000079s : 0.04% meta_unpack_prepare : 0.000783s : 0.37% pre_cconv : 0.000026s : 0.01% abstract_specialize : 0.064640s : 30.58% pack_expand : 0.000137s : 0.06% auto_monad : 0.001014s : 0.48% inline : 0.000032s : 0.01% pre_auto_parallel : 0.000037s : 0.02% pipeline_split : 0.000040s : 0.02% optimize.py_interpret_to_execute : 0.000225s : 0.11% optimize.rewriter_before_opt_a : 0.001108s : 0.52% optimize.opt_a.expand_dump_flag : 0.000019s : 0.01% optimize.opt_a.switch_simplify : 0.000347s : 0.16% optimize.opt_a.a_1 : 0.003446s : 1.63% optimize.opt_a.recompute_prepare : 0.000048s : 0.02% optimize.opt_a.updatestate_depend_eliminate : 0.000360s : 0.17% optimize.opt_a.updatestate_assign_eliminate : 0.000072s : 0.03% optimize.opt_a.updatestate_loads_eliminate : 0.000243s : 0.12% optimize.opt_a.parameter_eliminate : 0.000006s : 0.00% optimize.opt_a.a_2 : 0.000891s : 0.42% optimize.opt_a.accelerated_algorithm : 0.000056s : 0.03% optimize.opt_a.pynative_shard : 0.000083s : 0.04% optimize.opt_a.auto_parallel : 0.000009s : 0.00% optimize.opt_a.parallel : 0.000040s : 0.02% optimize.opt_a.merge_comm : 0.000027s : 0.01% optimize.opt_a.allreduce_fusion : 0.000015s : 0.01% optimize.opt_a.virtual_dataset : 0.000037s : 0.02% optimize.opt_a.get_grad_eliminate_ : 0.000031s : 0.01% optimize.opt_a.virtual_output : 0.000030s : 0.01% optimize.opt_a.merge_forward : 0.000049s : 0.02% optimize.opt_a.cell_reuse_recompute_pass : 0.000002s : 0.00% optimize.opt_a.cell_reuse_handle_not_recompute_node_pass : 0.000087s : 0.04% optimize.opt_a.meta_fg_expand : 0.000045s : 0.02% optimize.opt_a.after_resolve : 0.000045s : 0.02% optimize.opt_a.a_after_grad : 0.000040s : 0.02% optimize.opt_a.renormalize : 0.004275s : 2.02% optimize.opt_a.real_op_eliminate : 0.000041s : 0.02% optimize.opt_a.auto_monad_grad : 0.000008s : 0.00% optimize.opt_a.auto_monad_eliminator : 0.000180s : 0.08% optimize.opt_a.cse : 0.000253s : 0.12% optimize.opt_a.a_3 : 0.000278s : 0.13% optimize.py_interpret_to_execute_after_opt_a : 0.000033s : 0.02% optimize.slice_cell_reuse_recomputed_activation : 0.000003s : 0.00% optimize.rewriter_after_opt_a : 0.000891s : 0.42% optimize.convert_after_rewriter : 0.000033s : 0.02% optimize.order_py_execute_after_rewriter : 0.000021s : 0.01% optimize.opt_b.b_1 : 0.000420s : 0.20% optimize.opt_b.b_2 : 0.000018s : 0.01% optimize.opt_b.updatestate_depend_eliminate : 0.000023s : 0.01% optimize.opt_b.updatestate_assign_eliminate : 0.000021s : 0.01% optimize.opt_b.updatestate_loads_eliminate : 0.000024s : 0.01% optimize.opt_b.renormalize : 0.000035s : 0.02% optimize.opt_b.cse : 0.000092s : 0.04% optimize.cconv : 0.000035s : 0.02% optimize.opt_after_cconv.c_1 : 0.000059s : 0.03% optimize.opt_after_cconv.parameter_eliminate : 0.000001s : 0.00% optimize.opt_after_cconv.updatestate_depend_eliminate : 0.000024s : 0.01% optimize.opt_after_cconv.updatestate_assign_eliminate : 0.000024s : 0.01% optimize.opt_after_cconv.updatestate_loads_eliminate : 0.000024s : 0.01% optimize.opt_after_cconv.cse : 0.000079s : 0.04% optimize.opt_after_cconv.renormalize : 0.000032s : 0.02% optimize.remove_dup_value : 0.000100s : 0.05% optimize.tuple_transform.d_1 : 0.000118s : 0.06% optimize.tuple_transform.d_2 : 0.000043s : 0.02% optimize.tuple_transform.renormalize : 0.000030s : 0.01% optimize.add_cache_embedding : 0.000061s : 0.03% optimize.add_recomputation : 0.000267s : 0.13% optimize.cse_after_recomputation.cse : 0.000080s : 0.04% optimize.environ_conv : 0.000026s : 0.01% optimize.label_micro_interleaved_index : 0.000002s : 0.00% optimize.label_fine_grained_interleaved_index : 0.000002s : 0.00% optimize.assign_add_opt : 0.000032s : 0.01% optimize.slice_recompute_activation : 0.000002s : 0.00% optimize.micro_interleaved_order_control : 0.000002s : 0.00% optimize.full_micro_interleaved_order_control : 0.000002s : 0.00% optimize.comp_comm_scheduling : 0.000002s : 0.00% optimize.reorder_send_recv_between_fp_bp : 0.000002s : 0.00% optimize.comm_op_add_attrs : 0.000001s : 0.00% optimize.add_comm_op_reuse_tag : 0.000020s : 0.01% optimize.overlap_opt_shard_in_pipeline : 0.000013s : 0.01% optimize.grouped_pairwise_exchange_alltoall : 0.000013s : 0.01% optimize.overlap_recompute_and_grad_model_parallel : 0.000002s : 0.00% optimize.overlap_grad_matmul_and_grad_allreduce : 0.000021s : 0.01% optimize.split_matmul_comm_elemetwise : 0.000014s : 0.01% optimize.split_layernorm_comm : 0.000002s : 0.00% optimize.process_send_recv_for_ge : 0.000001s : 0.00% optimize.handle_group_info : 0.000001s : 0.00% auto_monad_reorder : 0.000143s : 0.07% get_jit_bprop_graph : 0.000023s : 0.01% eliminate_special_op_node : 0.000606s : 0.29% validate : 0.000129s : 0.06% distribtued_split : 0.000180s : 0.09% task_emit : 0.035949s : 17.01% execute : 0.000030s : 0.01% Time group info: ------[substitution.] 0.088082 1195 0.01% : 0.000007s : 5: substitution.depend_value_elim 0.01% : 0.000008s : 8: substitution.float_tuple_getitem_switch 97.85% : 0.086184s : 180: substitution.getattr_setattr_resolve 0.02% : 0.000019s : 40: substitution.graph_param_transform 1.55% : 0.001365s : 75: substitution.inline 0.01% : 0.000006s : 14: substitution.less_batch_normalization 0.01% : 0.000010s : 48: substitution.load_eliminater 0.24% : 0.000212s : 428: substitution.meta_unpack_prepare 0.01% : 0.000006s : 4: substitution.minmaximum_grad 0.00% : 0.000004s : 40: substitution.partial_unused_args_eliminate 0.01% : 0.000007s : 64: substitution.remove_not_recompute_node 0.01% : 0.000006s : 16: substitution.replace_old_param 0.02% : 0.000014s : 15: substitution.switch_simplify 0.03% : 0.000025s : 8: substitution.tuple_list_convert_item_index_to_positive 0.01% : 0.000010s : 8: substitution.tuple_list_get_item_const_eliminator 0.02% : 0.000015s : 8: substitution.tuple_list_get_item_depend_reorder 0.05% : 0.000043s : 15: substitution.tuple_list_get_item_eliminator 0.02% : 0.000015s : 8: substitution.tuple_list_get_set_item_eliminator 0.06% : 0.000050s : 104: substitution.updatestate_pure_node_eliminater 0.09% : 0.000077s : 107: substitution.updatestate_useless_node_eliminater ------[renormalize.] 0.004267 2 52.91% : 0.002258s : 1: renormalize.infer 47.09% : 0.002009s : 1: renormalize.specialize ------[replace.] 0.002719 256 0.47% : 0.000013s : 2: replace.depend_value_elim 77.64% : 0.002111s : 163: replace.getattr_setattr_resolve 16.87% : 0.000459s : 75: replace.inline 4.76% : 0.000129s : 15: replace.switch_simplify 0.27% : 0.000007s : 1: replace.tuple_list_get_item_eliminator ------[match.] 0.087330 256 0.00% : 0.000001s : 2: match.depend_value_elim 98.42% : 0.085947s : 163: match.getattr_setattr_resolve 1.56% : 0.001365s : 75: match.inline 0.02% : 0.000014s : 15: match.switch_simplify 0.00% : 0.000004s : 1: match.tuple_list_get_item_eliminator ------[func_graph_cloner_run.] 0.005389 106 69.44% : 0.003742s : 29: func_graph_cloner_run.FuncGraphClonerGraph 30.56% : 0.001647s : 77: func_graph_cloner_run.FuncGraphSpecializer ------[meta_graph.] 0.000000 0 ------[manager.] 0.000000 0 ------[pynative] 0.000000 0 ------[others.] 0.096997 122 1.26% : 0.001220s : 69: opt.transform.opt_a 0.42% : 0.000409s : 23: opt.transform.opt_b 93.06% : 0.090268s : 2: opt.transform.opt_resolve 0.74% : 0.000713s : 1: opt.transforms.meta_unpack_prepare 4.23% : 0.004100s : 20: opt.transforms.opt_a 0.06% : 0.000057s : 1: opt.transforms.opt_after_cconv 0.02% : 0.000017s : 1: opt.transforms.opt_b 0.16% : 0.000158s : 2: opt.transforms.opt_trans_graph 0.06% : 0.000056s : 3: opt.transforms.special_op_eliminate [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.103.846 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1385] Run] End [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.103.879 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:846] SaveCompiledGraph] Save compiled func graph(458_382_mindspore_train_dataset_helper__DataWrapper_construct_560) phase(eval.1704858064728525312.281469945694832.0)! [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.103.899 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:864] SaveCompiledGraph] End save compiled func graph! [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.103.912 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:942] CleanCompileRes] Clean compile resource start [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.106.868 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:956] CleanCompileRes] Clean compile resource end [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.106.904 [mindspore/ccsrc/pipeline/jit/ps/event_message_print.cc:37] PrintEventMessage] End compiling '_DataWrapper.construct'. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.106.918 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1039] CompileInner] Finish compiling. [WARNING] ME(164039:281472963273792,MainProcess):2024-01-10-11:41:05.107.485 [mindspore/parallel/_utils.py:259] You are suggested to use mindspore.context.set_auto_parallel_context(parameter_broadcast=True) or mindspore.common.set_seed() to share parameters among multi-devices. [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.108.521 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.108.573 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.108.667 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.108.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode 1 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.109.007 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[1], dtype=Int64, value=[32]), output index: 0 device address:0x4b06a460 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.109.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode false [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.109.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode 0 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.109.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Float32, value=0), output index: 0 device address:0x4f99eef0 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.109.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), output index: 0 device address:0x4f8ebbd0 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.109.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Float32, value=1), output index: 0 device address:0x4ed840f0 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.109.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Int64, value=10), output index: 0 device address:0x4f0b2ee0 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.109.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode -1 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.109.661 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10]), output index: 0 device address:0x4ae1ce70 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.109.827 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] GE(164039,python):2024-01-10-11:41:05.188.901 [graph_var_manager.cc:1424][EVENT]167201 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164039,python):2024-01-10-11:41:05.188.978 [graph_manager.cc:1248][EVENT]167201 PreRun:PreRun start: graph node size 2, session id 32, graph id 31, graph name online. [INFO] ATRACE(164039,python):2024-01-10-11:41:05.189.647 [atrace_api.c:28](tid:167201) AtraceCreate start [INFO] ATRACE(164039,python):2024-01-10-11:41:05.189.771 [trace_rb_log.c:84](tid:167201) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164039,python):2024-01-10-11:41:05.189.788 [atrace_api.c:32](tid:167201) AtraceCreate end [INFO] TDT(164039,python):2024-01-10-11:41:05.189.808 [client_manager.cpp:157][SetProfilingCallback][tid:167201] [TsdClient] set profiling callback success [INFO] GE(164039,python):2024-01-10-11:41:05.190.576 [parallel_partitioner.cc:165][EVENT]167201 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [16] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.190.613 [parallel_partitioner.cc:178][EVENT]167201 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.190.659 [graph_prepare.cc:1378][EVENT]167201 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.191.187 [graph_manager.cc:1050][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [545] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.191.214 [graph_manager.cc:1052][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.191.285 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.191.313 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.191.371 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [48] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.191.384 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.191.431 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.191.443 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.191.454 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.191.540 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.191.562 [graph_manager.cc:1054][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [335] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.191.801 [graph_manager.cc:1055][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [226] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.192.600 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:41:05.192.628 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.192.639 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.192.648 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [228] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.192.657 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.192.666 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:41:05.192.675 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.192.683 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.192.692 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.193.826 [graph_manager.cc:1056][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2006] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.193.896 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.193.916 [graph_prepare.cc:1982][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [48] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.194.243 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:41:05.194.264 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.275 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.284 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferShapePass is [142] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.293 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.302 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [4] [INFO] GE(164039,python):2024-01-10-11:41:05.194.310 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.319 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [6] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.327 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of InferValuePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.365 [graph_prepare.cc:1983][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [434] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.194.388 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.194.400 [graph_prepare.cc:1984][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.194.413 [graph_prepare.cc:1985][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.194.427 [graph_prepare.cc:1986][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.194.438 [graph_prepare.cc:1987][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.194.451 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.194.462 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.194.475 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.194.547 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.559 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.568 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.584 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.593 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.602 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.610 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.619 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.627 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.635 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.643 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.651 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.660 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.668 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.676 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.684 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.194.705 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.194.718 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.194.747 [graph_prepare.cc:1988][EVENT]167201 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [300] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.194.759 [graph_manager.cc:1065][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [893] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.206.545 [graph_manager.cc:1077][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11768] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.206.610 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.206.659 [graph_manager.cc:1080][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [79] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.208.998 [graph_manager.cc:1081][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [2323] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.037 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.062 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.073 [graph_manager.cc:1082][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [45] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.102 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.119 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.133 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.162 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [20] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.178 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.193 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.206 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.244 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [27] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.262 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.289 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.315 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.329 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.340 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.352 [graph_manager.cc:2700][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [255] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.437 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.209.453 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.209.464 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.209.475 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.209.484 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.209.492 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CastRemovePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.209.507 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.209.516 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.209.524 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.209.533 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.209.545 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [6] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.209.553 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.209.561 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [6] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.209.570 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.209.578 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.209.590 [graph_manager.cc:2741][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [218] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.602 [graph_manager.cc:2752][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.623 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.637 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.654 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.669 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.681 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.719 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.739 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.752 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.764 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.775 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.793 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [9] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.805 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.829 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.841 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.850 [graph_manager.cc:2810][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [230] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.876 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.209.886 [graph_manager.cc:2821][EVENT]167201 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [27] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.209.914 [graph_manager.cc:1087][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [824] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.210.042 [graph_manager.cc:1088][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [115] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.210.078 [graph_manager.cc:1089][EVENT]167201 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.210.097 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.210.112 [graph_manager.cc:1097][EVENT]167201 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164039,python):2024-01-10-11:41:05.210.132 [graph_manager.cc:3325][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.210.245 [engine_place.cc:144][EVENT]167201 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.210.259 [engine_place.cc:144][EVENT]167201 Run:The time cost of aicpu_ascend_kernel::CheckSupported is [43] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.210.328 [graph_manager.cc:3351][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [183] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.210.347 [graph_manager.cc:3364][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.210.410 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.210.426 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.210.552 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [114] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.210.581 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [15] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.210.620 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [27] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.210.651 [graph_manager.cc:3405][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [292] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.210.678 [graph_manager.cc:3412][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.206 [graph_manager.cc:3422][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [1513] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.237 [graph_manager.cc:3428][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.354 [graph_manager.cc:3467][EVENT]167201 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [97] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.371 [graph_manager.cc:3377][EVENT]167201 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [2012] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.386 [graph_manager.cc:1106][EVENT]167201 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [2260] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.398 [graph_manager.cc:1115][EVENT]167201 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:41:05.212.420 [graph_manager.cc:1130][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.450 [graph_manager.cc:1131][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.472 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.488 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.497 [graph_manager.cc:2837][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [33] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.555 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [8] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.212.567 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.212.576 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.212.585 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.212.594 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.212.602 [base_pass.cc:339][EVENT]167201 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [2] [INFO] GE(164039,python):2024-01-10-11:41:05.212.612 [graph_manager.cc:2864][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [98] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.623 [graph_manager.cc:2872][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.643 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.666 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.681 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.695 [compile_nodes_pass.cc:88][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.705 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.720 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.791 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [60] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.817 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.829 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.845 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.857 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.867 [graph_manager.cc:2927][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [227] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.882 [graph_manager.cc:2937][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.910 [graph_manager.cc:2943][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [18] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.212.926 [graph_manager.cc:2950][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [4] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.213.096 [graph_manager.cc:2958][EVENT]167201 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.213.126 [graph_manager.cc:1132][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [663] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.213.218 [graph_manager.cc:1135][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [79] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.213.254 [graph_manager.cc:2975][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.213.360 [graph_manager.cc:2981][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [92] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.213.376 [pass_manager.cc:82][EVENT]167201 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.213.387 [graph_manager.cc:2986][EVENT]167201 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.213.403 [graph_manager.cc:1136][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [169] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.213.505 [graph_manager.cc:3555][EVENT]167201 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [74] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.213.557 [engine_partitioner.cc:1139][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [12] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.213.571 [engine_partitioner.cc:1142][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [2] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.213.657 [engine_partitioner.cc:1148][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [76] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.213.679 [engine_partitioner.cc:1155][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [10] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.213.725 [engine_partitioner.cc:1164][EVENT]167201 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.213.748 [graph_builder.cc:865][EVENT]167201 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [215] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.213.823 [graph_builder.cc:288][EVENT]167201 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [56] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.213.967 [graph_builder.cc:293][EVENT]167201 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [127] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.214.148 [model_builder.cc:1133][EVENT]167201 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [88] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.214.396 [block_mem_assigner.cc:4069][EVENT]172209 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164039,python):2024-01-10-11:41:05.214.399 [block_mem_assigner.cc:4069][EVENT]172210 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164039,python):2024-01-10-11:41:05.214.794 [graph_mem_assigner.cc:2166][EVENT]167201 SetInputOffset:[IMAS]AfterAssignMemory : online_31 memoffset[132096], memtype[2], theory_min[264192], zero_copy[132096], total_size[132096], no_reuse[132096], streams[1], topo_mode[DFS], mop[], io_reuse[0:0], alloc_mode[] [INFO] GE(164039,python):2024-01-10-11:41:05.214.880 [model_builder.cc:1144][EVENT]167201 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [711] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.214.903 [model_builder.cc:1152][EVENT]167201 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [8] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.214.919 [model_builder.cc:1157][EVENT]167201 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.215.029 [model_builder.cc:1167][EVENT]167201 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [98] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.215.047 [model_builder.cc:1174][EVENT]167201 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.215.067 [model_builder.cc:1180][EVENT]167201 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [7] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.215.108 [model_builder.cc:1184][EVENT]167201 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.215.126 [graph_builder.cc:304][EVENT]167201 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [1136] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:41:05.215.307 [logger.cc:1071] 167201 ModelBindStream: model_id=64, stream_id=1345, flag=0. [INFO] GE(164039,python):2024-01-10-11:41:05.215.372 [task_generator.cc:804][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [5] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.215.427 [task_generator.cc:805][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [42] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.215.905 [task_generator.cc:814][EVENT]167201 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [464] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.215.921 [task_generator.cc:954][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [554] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.215.979 [task_generator.cc:967][EVENT]167201 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [35] micro second. [INFO] RUNTIME(164039,python):2024-01-10-11:41:05.216.000 [logger.cc:1084] 167201 ModelUnbindStream: model_id=64, stream_id=1345, [INFO] GE(164039,python):2024-01-10-11:41:05.216.058 [graph_builder.cc:310][EVENT]167201 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [918] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.216.162 [graph_manager.cc:1152][EVENT]167201 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2738] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.216.181 [graph_manager.cc:1164][EVENT]167201 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164039,python):2024-01-10-11:41:05.216.213 [graph_manager.cc:1271][EVENT]167201 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [25727] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.216.223 [graph_manager.cc:1272][EVENT]167201 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164039,python):2024-01-10-11:41:05.216.534 [atrace_api.c:93](tid:167201) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:41:05.216.552 [atrace_api.c:95](tid:167201) AtraceDestroy end [INFO] GE(164039,python):2024-01-10-11:41:05.217.169 [model_introduction.cc:236][EVENT]167201 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(164039,python):2024-01-10-11:41:05.217.190 [model_introduction.cc:294][EVENT]167201 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(164039,python):2024-01-10-11:41:05.217.205 [model_introduction.cc:366][EVENT]167201 Data:model io_info size:116 [INFO] GE(164039,python):2024-01-10-11:41:05.220.645 [graph_converter.cc:838][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1267] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.220.825 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [135] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.221.227 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [379] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.221.310 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [59] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.221.326 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [77] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.221.370 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [31] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.221.414 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.221.446 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of ZeroCopy is [19] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.221.515 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CEM is [57] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.221.580 [copy_flow_launch_fuse.cc:395][EVENT]167201 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [49] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.221.592 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [62] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.221.622 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [21] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.221.648 [base_optimizer.cc:70][EVENT]167201 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.221.663 [graph_converter.cc:849][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [980] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.221.879 [graph_converter.cc:853][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [205] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.222.503 [graph_converter.cc:857][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [603] micro second. [INFO] GE(164039,python):2024-01-10-11:41:05.222.613 [graph_converter.cc:862][EVENT]167201 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [86] micro second. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:05.227.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 1 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.227.461 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 1, execution time: 118.687 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.227.611 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.227.699 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.227.731 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.227.758 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.228.996 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.229.056 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.229.092 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:05.229.302 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.231.697 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 2 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.231.798 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 2, execution time: 2.63754 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.231.873 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.231.923 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.231.953 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.231.976 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.232.751 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.232.795 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.232.821 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.232.962 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.234.939 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.235.020 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 3, execution time: 2.1437 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.235.090 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.235.138 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.235.164 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.235.185 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.235.902 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.235.945 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.235.973 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.236.107 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.237.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 4 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.238.033 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 4, execution time: 2.0077 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.238.104 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.238.148 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.238.176 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.238.197 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.238.920 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.238.961 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.238.987 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.239.118 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.241.043 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 5 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.241.119 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 5, execution time: 2.082 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.241.189 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.241.232 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.241.274 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.241.297 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.242.026 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.242.071 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.242.096 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.242.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.244.079 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 6 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.244.155 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 6, execution time: 2.0063 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.244.228 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.244.271 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.244.296 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.244.316 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.245.032 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.245.075 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.245.100 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.245.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.247.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 7 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.247.198 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 7, execution time: 2.04701 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.247.265 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.247.305 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.247.331 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.247.351 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.248.071 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.248.113 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.248.139 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.248.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.250.120 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 8 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.250.212 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 8, execution time: 2.02247 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.250.280 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.250.321 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.250.349 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.250.369 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.251.083 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.251.126 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.251.150 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.251.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:05.253.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 9 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.253.195 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 9, execution time: 1.99562 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.253.265 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.253.310 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.253.337 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.253.358 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.254.108 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.254.153 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.254.179 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.254.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.256.192 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 10 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.256.281 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 10, execution time: 2.05261 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.256.350 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.256.394 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.256.423 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.256.443 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.257.155 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.257.197 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.257.225 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.257.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.259.243 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 11 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.259.319 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 11, execution time: 2.03771 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.259.389 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.259.431 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.259.459 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.259.481 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.260.195 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.260.237 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.260.264 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.260.399 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.262.326 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 12 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.262.410 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 12, execution time: 2.09428 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.262.480 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.262.521 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.262.545 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.262.566 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.263.294 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.263.335 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.263.364 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.263.498 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.265.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 13 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.265.437 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 13, execution time: 2.02278 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.265.504 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.265.545 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.265.571 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.265.592 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.266.348 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.266.391 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.266.419 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.266.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.268.442 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 14 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.268.524 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 14, execution time: 2.05519 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.268.594 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.268.634 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.268.660 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.268.679 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.269.400 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.269.442 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.269.467 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.269.597 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.271.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 15 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.271.529 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 15, execution time: 2.01378 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.271.599 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.271.640 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.271.665 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.271.684 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.272.394 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.272.436 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.272.464 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.272.598 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.274.460 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 16 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.274.542 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 16, execution time: 2.02468 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.274.611 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.274.652 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.274.678 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.274.699 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.275.411 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.275.453 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.275.481 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.275.615 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.277.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 17 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.277.549 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 17, execution time: 2.01516 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.277.618 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.277.659 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.277.685 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.277.723 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.278.448 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.278.492 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.278.517 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.278.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.280.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 18 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.280.622 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 18, execution time: 2.0549 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.280.691 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.280.731 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.280.757 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.280.778 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.281.486 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.281.527 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.281.553 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.281.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.283.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 19 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.283.646 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 19, execution time: 2.04297 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.283.716 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.283.760 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.283.787 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.283.808 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.284.524 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.284.565 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.284.589 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.284.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:05.286.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 20 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.286.716 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 20, execution time: 2.07559 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.286.797 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.286.842 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.286.871 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.286.893 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.287.611 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.287.655 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.287.681 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.287.808 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.289.633 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 21 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.289.716 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 21, execution time: 1.98536 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.289.791 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.289.836 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.289.866 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.289.888 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.290.607 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.290.650 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.290.675 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.290.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:05.292.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 22 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.292.609 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 22, execution time: 1.88183 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.292.690 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.292.732 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.292.756 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.292.777 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.293.508 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.293.552 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.293.578 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.293.728 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.295.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 23 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.295.712 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 23, execution time: 2.08329 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.295.783 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.295.828 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.295.855 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.295.875 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.296.586 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.296.627 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.296.655 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.296.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.298.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 24 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.298.752 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 24, execution time: 2.0456 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.298.836 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.298.880 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.298.909 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.298.929 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.299.647 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.299.689 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.299.717 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.299.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.301.649 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 25 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.301.735 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 25, execution time: 1.96371 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.301.805 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.301.848 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.301.875 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.301.898 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.302.622 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.302.665 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.302.690 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.302.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.304.720 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 26 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.304.794 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 26, execution time: 2.05161 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.304.877 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.304.922 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.304.951 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.304.974 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.305.709 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.305.755 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.305.781 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.305.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.307.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 27 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.307.843 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 27, execution time: 2.01259 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.307.910 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.307.955 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.307.984 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.308.004 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.308.716 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.308.757 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.308.787 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.308.922 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.310.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 28 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.310.866 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 28, execution time: 2.02419 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.310.950 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.310.991 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.311.017 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.311.038 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.311.773 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.311.815 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.311.841 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.311.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.313.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 29 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.314.015 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 29, execution time: 2.12316 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.314.085 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.314.126 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.314.152 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.314.173 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.314.892 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.314.934 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.314.959 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.315.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.316.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 30 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.317.001 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 30, execution time: 1.99243 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.317.077 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.317.118 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.317.144 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.317.166 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.317.899 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.317.943 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.317.967 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.318.102 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.319.865 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 31 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.319.941 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 31, execution time: 1.91973 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.320.008 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.320.050 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.320.077 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.320.097 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.320.812 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.320.854 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.320.880 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.321.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.322.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 32 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.322.865 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 32, execution time: 1.93245 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.322.936 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.322.987 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.323.011 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.323.032 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.323.749 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.323.791 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.323.815 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.323.943 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.325.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 33 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.325.986 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 33, execution time: 2.12106 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.326.053 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.326.096 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.326.125 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.326.145 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.326.856 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.326.897 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.326.923 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.327.052 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.328.865 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 34 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.328.936 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 34, execution time: 1.96264 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.329.005 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.329.058 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.329.084 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.329.104 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.329.838 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.329.882 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.329.907 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.330.036 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.331.888 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 35 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.331.962 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 35, execution time: 2.00371 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.332.031 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.332.072 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.332.097 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.332.116 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.332.834 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.332.876 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.332.901 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.333.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.334.898 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 36 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.334.986 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 36, execution time: 2.03496 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.335.056 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.335.109 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.335.137 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.335.157 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.335.882 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.335.925 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.335.953 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.336.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1bf470f0,python):2024-01-10-11:41:05.337.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 37 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.338.016 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 37, execution time: 2.01005 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.338.088 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.338.133 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.338.162 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.338.185 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.338.899 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.338.942 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.338.968 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.339.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.340.922 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 38 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.340.992 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 38, execution time: 1.97284 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.341.058 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.341.111 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.341.140 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.341.163 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.341.893 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.341.936 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.341.965 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.342.097 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.343.879 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 39 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.343.954 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 39, execution time: 1.93818 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.344.022 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.344.063 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.344.089 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.344.110 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.344.821 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.344.863 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.344.887 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,fffe1cf490f0,python):2024-01-10-11:41:05.345.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,fffe1c7480f0,python):2024-01-10-11:41:05.346.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 40 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.347.006 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 40, execution time: 2.07167 ms in multi thread or not: 1. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.347.075 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.347.115 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.347.150 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.347.170 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.347.891 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.347.933 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.347.958 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.348.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.350.058 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 41 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.350.135 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 41, execution time: 2.12699 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.350.208 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.350.253 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.350.280 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.350.301 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.351.029 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.351.071 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.351.096 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.351.204 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.353.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 42 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.353.098 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 42, execution time: 1.94997 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.353.170 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.353.217 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.353.268 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.353.290 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.354.051 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.354.095 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.354.121 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.354.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.356.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 43 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.356.123 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 43, execution time: 1.94953 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.356.194 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.356.241 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.356.271 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.356.295 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.357.023 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.357.066 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.357.091 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.357.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.359.059 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 44 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.359.133 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 44, execution time: 1.99401 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.359.205 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.359.250 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.359.287 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.359.310 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.360.042 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.360.084 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.360.111 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.360.217 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.362.072 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 45 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.362.150 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 45, execution time: 1.98856 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.362.223 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.362.271 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.362.298 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.362.319 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.363.058 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.363.100 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.363.125 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.363.231 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.365.059 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 46 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.365.132 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 46, execution time: 1.95665 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.365.202 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.365.247 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.365.284 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.365.309 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.366.054 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.366.098 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.366.126 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.366.232 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.368.035 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 47 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.368.109 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 47, execution time: 1.93244 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.368.181 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.368.228 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.368.256 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.368.277 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.369.003 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.369.045 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.369.075 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.369.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.371.019 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 48 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.371.093 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 48, execution time: 1.96797 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.371.166 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.371.213 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.371.241 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.371.275 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.372.008 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.372.050 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.372.076 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.372.183 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.374.061 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 49 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.374.136 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 49, execution time: 2.00795 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.374.207 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.374.252 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.374.283 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.374.304 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.375.049 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.375.091 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.375.117 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.375.222 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.377.030 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 50 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.377.104 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 50, execution time: 1.93693 ms in multi thread or not: 0. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.377.126 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:767] SetActorExecutionStrategy] Multi thread execution time cost: 1.99673 ms, single thread execution time cost: 1.9811 ms, decide to use multi thread execution or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.377.189 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.377.239 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.377.267 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.377.287 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.378.149 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.378.193 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.378.217 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.378.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.380.139 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 51 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.380.212 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 51, execution time: 1.94515 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.380.282 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.380.327 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.380.355 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.380.376 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.381.108 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.381.150 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.381.176 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.381.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.383.112 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 52 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.383.190 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 52, execution time: 1.96346 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.383.265 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.383.319 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.383.347 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.383.373 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.384.107 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.384.149 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.384.176 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.384.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.386.148 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 53 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.386.223 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 53, execution time: 1.99839 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.386.293 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.386.337 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.386.366 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.386.388 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.387.116 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.387.157 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.387.182 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.387.287 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.389.086 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 54 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.389.159 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 54, execution time: 1.92692 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.389.231 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.389.277 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.389.314 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.389.337 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.390.080 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.390.125 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.390.152 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.390.259 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.392.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 55 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.392.133 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 55, execution time: 1.92879 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.392.204 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.392.251 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.392.281 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.392.303 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.393.027 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.393.069 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.393.097 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.393.200 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.395.027 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 56 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.395.102 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 56, execution time: 1.95495 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.395.174 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.395.221 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.395.261 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.395.284 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.396.012 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.396.055 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.396.083 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.396.192 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.398.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 57 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.398.127 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 57, execution time: 1.9911 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.398.199 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.398.244 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.398.271 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.398.294 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.399.021 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.399.063 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.399.088 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.399.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.400.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 58 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.401.063 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 58, execution time: 1.92531 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.401.136 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.401.180 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.401.217 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.401.238 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.401.984 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.402.028 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.402.055 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.402.162 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.403.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 59 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.404.052 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 59, execution time: 1.94426 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.404.123 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.404.167 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.404.194 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.404.215 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.404.944 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.404.986 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.405.012 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.405.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.406.970 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 60 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.407.046 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 60, execution time: 1.98393 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.407.120 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.407.165 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.407.201 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.407.223 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.407.951 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.407.994 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.408.019 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.408.127 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.409.964 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 61 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.410.042 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 61, execution time: 1.97179 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.410.116 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.410.164 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.410.192 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.410.213 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.410.945 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.410.989 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.411.015 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.411.122 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.412.944 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 62 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.413.017 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 62, execution time: 1.95067 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.413.090 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.413.137 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.413.168 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.413.202 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.413.962 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.414.004 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.414.030 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.414.135 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.415.961 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 63 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.416.034 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 63, execution time: 1.95333 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.416.107 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.416.152 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.416.182 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.416.206 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.416.935 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.416.978 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.417.004 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.417.108 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.418.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 64 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.419.028 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 64, execution time: 1.97437 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.419.099 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.419.144 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.419.171 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.419.201 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.419.928 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.419.970 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.419.995 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.420.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.421.933 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 65 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.422.008 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 65, execution time: 1.96248 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.422.079 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.422.124 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.422.152 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.422.173 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.422.915 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.422.957 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.422.982 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.423.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.424.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 66 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.424.991 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 66, execution time: 1.95699 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.425.067 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.425.113 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.425.144 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.425.178 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.425.933 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.425.976 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.426.004 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.426.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.427.942 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 67 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.428.018 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 67, execution time: 1.96083 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.428.092 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.428.138 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.428.169 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.428.193 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.428.921 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.428.964 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.428.993 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.429.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.430.939 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 68 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.431.014 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 68, execution time: 1.96775 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.431.084 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.431.131 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.431.162 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.431.193 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.431.927 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.431.969 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.431.994 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.432.099 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.433.936 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 69 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.434.010 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 69, execution time: 1.96696 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.434.083 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.434.128 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.434.156 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.434.177 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.434.905 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.434.947 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.434.974 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.435.080 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.436.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 70 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.436.995 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 70, execution time: 1.9706 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.437.067 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.437.111 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.437.139 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.437.160 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.437.954 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.437.997 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.438.025 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.438.134 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.439.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 71 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.440.027 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 71, execution time: 1.94777 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.440.098 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.440.142 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.440.170 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.440.191 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.440.919 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.440.961 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.440.987 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.441.091 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.442.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 72 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.443.029 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 72, execution time: 1.9911 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.443.100 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.443.145 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.443.171 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.443.193 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.443.930 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.443.971 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.443.996 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.444.102 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.445.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 73 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.446.029 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 73, execution time: 1.98291 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.446.099 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.446.142 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.446.169 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.446.190 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.446.926 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.446.967 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.446.993 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.447.100 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.448.935 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 74 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.449.011 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 74, execution time: 1.96664 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.449.081 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.449.126 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.449.154 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.449.176 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.449.941 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.449.983 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.450.008 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.450.115 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.451.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 75 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.452.007 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 75, execution time: 1.94754 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.452.077 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.452.122 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.452.149 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.452.170 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.452.895 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.452.936 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.452.962 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.453.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.454.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 76 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.454.981 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 76, execution time: 1.96814 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.455.052 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.455.097 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.455.125 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.455.147 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.455.890 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.455.932 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.455.958 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.456.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.457.942 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 77 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.458.017 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 77, execution time: 2.00923 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.458.090 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.458.135 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.458.166 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.458.189 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164039,ffff87fd6440,python):2024-01-10-11:41:05.458.927 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.458.969 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.458.995 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.459.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.460.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 78 [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:05.461.007 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 78, execution time: 1.96088 ms in multi thread or not: 0. [INFO] VM(164039,ffff87fd6440,python):2024-01-10-11:41:05.461.080 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.461.126 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.461.154 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.461.175 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty Accuracy: 0.07732371794871795 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.741.000 [mindspore/ccsrc/pipeline/jit/ps/init.cc:515] operator()] Start register... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.741.089 [mindspore/ccsrc/pipeline/jit/ps/init.cc:519] operator()] Start mindspore.profiler... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.741.158 [mindspore/ccsrc/pipeline/jit/ps/init.cc:527] operator()] Start EmbeddingCacheScheduler... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.741.185 [mindspore/ccsrc/pipeline/jit/ps/init.cc:534] operator()] Start releasing dataset handles... [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:05.741.241 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164039,ffff87fd6440,python):2024-01-10-11:41:05.741.358 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.748.315 [mindspore/ccsrc/pipeline/jit/ps/init.cc:537] operator()] End release dataset handles. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:05.748.365 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2393] FinalizeCluster] Start finalize the cluster instance. [INFO] DISTRIBUTED(164039,ffff064910f0,python):2024-01-10-11:41:06.513.681 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:301] Heartbeat] The heartbeat thread is finished. [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:41:06.514.271 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:131] Finalize] The compute graph node has been unregistered successfully. [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:41:06.514.373 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:533] Finalize] Delete send event loop [INFO] DISTRIBUTED(164039,ffff07c940f0,python):2024-01-10-11:41:06.514.466 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:41:06.514.632 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:209] Finalize] Stop loop succ [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:41:06.514.652 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:540] Finalize] Delete recv event loop [INFO] DISTRIBUTED(164039,ffff084950f0,python):2024-01-10-11:41:06.514.718 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:41:06.514.876 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:209] Finalize] Stop loop succ [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:41:06.514.890 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:554] Finalize] Delete connection pool. [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:41:06.514.942 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:533] Finalize] Delete send event loop [INFO] DISTRIBUTED(164039,ffff06c920f0,python):2024-01-10-11:41:06.515.012 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:41:06.515.139 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:209] Finalize] Stop loop succ [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:41:06.515.158 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:540] Finalize] Delete recv event loop [INFO] DISTRIBUTED(164039,ffff074930f0,python):2024-01-10-11:41:06.515.198 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:41:06.515.326 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:209] Finalize] Stop loop succ [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:41:06.515.340 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:554] Finalize] Delete connection pool. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:06.515.359 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2396] FinalizeCluster] End finalize the cluster instance. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:06.515.387 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2335] ClearResAtexit] Pipeline clear all resource [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:06.515.481 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:290] RecordExitStatus] Status record: system exit. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:06.519.618 [mindspore/ccsrc/runtime/graph_scheduler/rpc_node_scheduler.cc:220] Clear] Start finalizing tcp server and client for rpc actors. [INFO] RUNTIME_FRAMEWORK(164039,ffff87fd6440,python):2024-01-10-11:41:06.519.664 [mindspore/ccsrc/runtime/graph_scheduler/rpc_node_scheduler.cc:230] Clear] End finalizing tcp server and client for rpc actors. [INFO] ME(164039,ffff87fd6440,python):2024-01-10-11:41:06.520.117 [mindspore/core/mindrt/src/actor/actormgr.cc:153] Finalize] mindrt Actors finish exiting. [INFO] ME(164039,ffff87fd6440,python):2024-01-10-11:41:06.520.135 [mindspore/core/mindrt/src/actor/actormgr.cc:156] Finalize] mindrt Threads finish exiting. [INFO] ME(164039,ffff87fd6440,python):2024-01-10-11:41:06.538.030 [mindspore/core/mindrt/src/actor/actormgr.cc:167] Finalize] mindrt IOMGRS finish exiting. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:06.538.903 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2207] ClearResPart1] Start Finalize StreamSynchronizer... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:06.538.938 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2209] ClearResPart1] End Finalize StreamSynchronizer... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:06.540.315 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:829] ClearRes] Clean executor resource! [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:06.540.341 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2223] ClearResPart2] Start clear PyNativeExecutor... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:06.540.636 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2225] ClearResPart2] End clear PyNativeExecutor. [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:41:06.540.686 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:179] ClearGraph] Remove all graphs in GraphManager [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:06.548.814 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2241] ClearResPart2] Start clear kernel runtime... [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:06.548.850 [mindspore/ccsrc/runtime/device/kernel_runtime_manager.cc:25] ClearRuntimeResource] Release device Ascend_0 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:06.548.867 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:240] ReleaseDeviceRes] Ascend finalize start [INFO] HCCL(164039,python):2024-01-10-11:41:06.548.983 [op_base.cc:1312][164039]com is not global com [INFO] HCCP(164039,python):2024-01-10-11:41:06.549.190 [ra_host.c:1795]tid:164039,ra_socket_white_list_del(1795) : Input parameters: phy_id[0], local_ip[0.0.0.0], num[1] [INFO] HCCP(164039,python):2024-01-10-11:41:06.549.997 [ra_host.c:863]tid:164039,ra_socket_batch_close(863) : Input parameters: [0]th, phy_id[0], local_ip[0.0.0.0] [INFO] HCCP(164039,python):2024-01-10-11:41:06.593.087 [ra_host.c:863]tid:164039,ra_socket_batch_close(863) : Input parameters: [0]th, phy_id[0], local_ip[0.0.0.0] [INFO] HCCP(164039,python):2024-01-10-11:41:06.699.201 [ra_host.c:863]tid:164039,ra_socket_batch_close(863) : Input parameters: [0]th, phy_id[0], local_ip[0.0.0.0] [INFO] HCCP(164039,python):2024-01-10-11:41:06.699.230 [ra_host.c:863]tid:164039,ra_socket_batch_close(863) : Input parameters: [1]th, phy_id[0], local_ip[0.0.0.0] [INFO] HCCP(164039,python):2024-01-10-11:41:06.699.242 [ra_host.c:863]tid:164039,ra_socket_batch_close(863) : Input parameters: [2]th, phy_id[0], local_ip[0.0.0.0] [INFO] HCCP(164039,python):2024-01-10-11:41:06.801.030 [ra_host.c:1795]tid:164039,ra_socket_white_list_del(1795) : Input parameters: phy_id[0], local_ip[0.0.0.0], num[1] [INFO] HCCP(164039,python):2024-01-10-11:41:06.801.142 [ra_host.c:1795]tid:164039,ra_socket_white_list_del(1795) : Input parameters: phy_id[0], local_ip[0.0.0.0], num[1] [INFO] HCCP(164039,python):2024-01-10-11:41:06.805.242 [ra_host.c:941]tid:164039,ra_socket_listen_stop(941) : Input parameters: [0]th, phy_id[0], local_ip[0.0.0.0] [INFO] HCCP(164039,python):2024-01-10-11:41:06.805.361 [ra_host.c:525]tid:164039,ra_socket_deinit(525) : Input parameters: phy_id[0] family[2] local_ip[0.0.0.0] [INFO] HCCP(164039,python):2024-01-10-11:41:06.805.437 [ra_host.c:349]tid:164039,ra_deinit(349) : Input parameters: phy_id[0], nic_position:[1] [INFO] HCCP(164039,python):2024-01-10-11:41:06.805.450 [ra_hdc.c:1535]tid:164039,ra_hdc_deinit(1535) : hdc deinit start! phy_id[0] [INFO] HCCP(164039,python):2024-01-10-11:41:06.805.565 [ra_hdc.c:1570]tid:164039,ra_hdc_deinit(1570) : hdc deinit OK! phy_id[0] [INFO] ATRACE(164039,python):2024-01-10-11:41:06.805.656 [atrace_api.c:93](tid:164039) AtraceDestroy start [INFO] ATRACE(164039,python):2024-01-10-11:41:06.805.681 [atrace_api.c:95](tid:164039) AtraceDestroy end [INFO] HCCL(164039,python):2024-01-10-11:41:06.826.582 [op_base.cc:1332][164039]op_base comm destroy complete,take time [277639]us, rankNum[0], rank[4294967295], deviceLogicId[0] [INFO] HCCL_ADPT(164039,ffff87fd6440,python):2024-01-10-11:41:06.826.654 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:260] FinalizeHccl] Start destroy hccl adapter for GRAPH_MODE [INFO] HCCL_ADPT(164039,ffff87fd6440,python):2024-01-10-11:41:06.826.679 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:548] FinalizeHcclExec] Start finalize hccl exec. [INFO] HCCL(164039,python):2024-01-10-11:41:06.831.201 [hcom_executor.cc:43][164039][Finalize][HcomExecutor]Hcom Excutor Finalize end. ret[0] [INFO] HCCL_ADPT(164039,ffff87fd6440,python):2024-01-10-11:41:06.831.244 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:556] FinalizeHcclExec] HcclExec destroy success [INFO] HCCL_ADPT(164039,ffff87fd6440,python):2024-01-10-11:41:06.831.266 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:388] FinalizeKernelInfoStore] Start destroy hccl kernel info store. [INFO] HCCL_ADPT(164039,ffff87fd6440,python):2024-01-10-11:41:06.831.315 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:410] FinalizeKernelInfoStore] Destroy hccl kernel info store success. [INFO] HCCL_ADPT(164039,ffff87fd6440,python):2024-01-10-11:41:06.831.330 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:437] FinalizeHcclComm] Start finalize hccl comm. [INFO] HCCL_ADPT(164039,ffff87fd6440,python):2024-01-10-11:41:06.831.494 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:273] FinalizeHccl] Destroy hccl adapter success. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:06.831.511 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:710] DestroyHccl] Hccl destroy successful. [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:06.831.565 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:783] operator()] Common mem pool info: Total allocated mem:1024M, peak used mem:4M, in used mem:0M, total idle mem:1023M. Block unit size:1024M, block counts:1, block[0] block size:1024M idle size:1023M [INFO] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:06.831.589 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:783] operator()] Persistent mem pool info: Total allocated mem:1024M, peak used mem:0M, in used mem:0M, total idle mem:1023M. Block unit size:1024M, block counts:1, block[0] block size:1024M idle size:1023M [WARNING] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:06.831.604 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:793] DumpDynamicMemPoolStateInfo] The dynamic memory pool total allocated mem:2048M, peak used mem:5M, in used mem:0M, total idle mem:71812935M, total eager free mem:0M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [WARNING] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:08.320.168 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:142] DeInitialize] Ascend Memory Adapter deinitialize success, statistics: Device HBM memory size: 32768M MindSpore Used memory size: 30680M MindSpore memory base address: 0x124100000000 Total Static Memory size: 2048M Total Dynamic memory size: 0M Dynamic memory size of this graph: 0M [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:08.342.013 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:274] ReleaseDeviceRes] Ascend finalize end [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:08.342.066 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2243] ClearResPart2] End clear kernel runtime. [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:41:08.342.087 [mindspore/ccsrc/distributed/collective/collective_manager.cc:379] Finalize] Begin finalize collective manager. [INFO] DISTRIBUTED(164039,fffe9effd0f0,python):2024-01-10-11:41:08.342.217 [mindspore/ccsrc/distributed/collective/collective_manager.cc:358] operator()] Start finalizing host communication lib. [INFO] DISTRIBUTED(164039,fffe9effd0f0,python):2024-01-10-11:41:08.342.254 [mindspore/ccsrc/distributed/collective/collective_manager.cc:362] operator()] End finalizing host communication lib. [INFO] DISTRIBUTED(164039,fffe9effd0f0,python):2024-01-10-11:41:08.342.267 [mindspore/ccsrc/distributed/collective/collective_manager.cc:367] operator()] Start finalizing device communication lib. [INFO] DISTRIBUTED(164039,fffe9effd0f0,python):2024-01-10-11:41:08.342.279 [mindspore/ccsrc/distributed/collective/collective_manager.cc:371] operator()] End finalizing device communication lib. [INFO] DISTRIBUTED(164039,ffff87fd6440,python):2024-01-10-11:41:08.342.302 [mindspore/ccsrc/distributed/collective/collective_manager.cc:386] Finalize] End finalize collective manager. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:08.342.318 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2258] ClearResPart2] Start clear device context... [INFO] ME(164039,ffff87fd6440,python):2024-01-10-11:41:08.342.332 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:469] ClearDeviceContexts] Release device Ascend_0 [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:41:08.342.364 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:264] DeleteGraphRunner] Delete GraphRunner success [TRACE] GE(164039,python):2024-01-10-11:41:08.342.384 [status:INIT] [ge_api.cc:463]164039 ~Session:Start to destruct session. [TRACE] GE(164039,python):2024-01-10-11:41:08.342.412 [status:RUNNING] [ge_api.cc:475]164039 ~Session:Session id is 0 [TRACE] GE(164039,python):2024-01-10-11:41:08.342.422 [status:RUNNING] [ge_api.cc:476]164039 ~Session:Destroying session [TRACE] GE(164039,python):2024-01-10-11:41:08.343.248 [status:STOP] [ge_api.cc:491]164039 ~Session:Session Destructor finished [INFO] GE_ADPT(164039,ffff87fd6440,python):2024-01-10-11:41:08.343.287 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:235] DeleteGeSession] Delete Ge Session success [TRACE] GE(164039,python):2024-01-10-11:41:08.343.307 [status:INIT] [ge_api.cc:301]164039 GEFinalize:GEFinalize start [INFO] GE(164039,python):2024-01-10-11:41:08.343.406 [execution_runtime.cc:80][EVENT]164039 FinalizeExecutionRuntime:Execution runtime finalize begin. [INFO] GE(164039,python):2024-01-10-11:41:08.343.426 [execution_runtime.cc:92][EVENT]164039 FinalizeExecutionRuntime:Execution runtime finalized. [TRACE] GE(164039,python):2024-01-10-11:41:08.343.436 [status:RUNNING] [ge_api.cc:313]164039 GEFinalize:Finalizing environment [INFO] TUNE(164039,python):2024-01-10-11:41:08.692.545 [cann_kb_pyfunc_mgr.cpp:127][CANNKB][Tid:164039]"CannKbPyfuncMgr: enter PyObjectDeinit function, reference_[1]" [INFO] TUNE(164039,python):2024-01-10-11:41:08.692.604 [cann_kb_pyfunc_mgr.cpp:138][CANNKB][Tid:164039]"CannKbPyfuncMgr: PyObjectDeinit function end successfully!" [INFO] GE(164039,python):2024-01-10-11:41:08.724.136 [gelib.cc:324][EVENT]164039 SystemFinalize:Online infer finalize GELib success. [TRACE] GE(164039,python):2024-01-10-11:41:09.114.168 [status:STOP] [ge_api.cc:341]164039 GEFinalize:GEFinalize finished [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:09.114.271 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:317] CloseTsd] Start to close tsd, ref = 1 [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:09.114.884 [mindspore/ccsrc/plugin/device/ascend/hal/device/tensorprint_utils.cc:449] DestroyTensorPrintThread] Succeed stop acl data channel for host queue [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:09.115.311 [mindspore/ccsrc/plugin/device/ascend/hal/device/tensorprint_utils.cc:375] JoinAclPrintThread] join acl tdt host receive process [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:09.115.358 [mindspore/ccsrc/plugin/device/ascend/hal/device/tensorprint_utils.cc:463] DestroyTensorPrintThread] Succeed destroy acl channel [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:09.115.382 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:119] DestoryHandler] The thread of ms_histogram_summary channel is being destroyed. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:09.115.396 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:115] ~MbufDataHandler] Channel ms_histogram_summary begins the destruction process. [INFO] DEVICE(164039,fffebef100f0,python):2024-01-10-11:41:09.134.943 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:67] ~ScopeAclTdtDataset] AcltdtDestroyDataset succeed. [INFO] DEVICE(164039,fffebd70d0f0,python):2024-01-10-11:41:09.135.120 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:67] ~ScopeAclTdtDataset] AcltdtDestroyDataset succeed. [INFO] DEVICE(164039,fffebdf0e0f0,python):2024-01-10-11:41:09.135.163 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:67] ~ScopeAclTdtDataset] AcltdtDestroyDataset succeed. [INFO] DEVICE(164039,fffebcf0c0f0,python):2024-01-10-11:41:09.135.205 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:67] ~ScopeAclTdtDataset] AcltdtDestroyDataset succeed. [INFO] DEVICE(164039,fffebe70f0f0,python):2024-01-10-11:41:09.135.225 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:67] ~ScopeAclTdtDataset] AcltdtDestroyDataset succeed. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:09.135.681 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:119] DestoryHandler] The thread of ms_scalar_summary channel is being destroyed. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:09.135.701 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:115] ~MbufDataHandler] Channel ms_scalar_summary begins the destruction process. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:09.135.981 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:119] DestoryHandler] The thread of ms_image_summary channel is being destroyed. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:09.135.998 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:115] ~MbufDataHandler] Channel ms_image_summary begins the destruction process. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:09.136.221 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:119] DestoryHandler] The thread of ms_tensor_summary channel is being destroyed. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:09.136.235 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:115] ~MbufDataHandler] Channel ms_tensor_summary begins the destruction process. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:09.136.440 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:119] DestoryHandler] The thread of ms_tensor_dump channel is being destroyed. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:09.136.454 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:115] ~MbufDataHandler] Channel ms_tensor_dump begins the destruction process. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:09.136.709 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:337] CloseTsd] Call aclrtResetDevice, destroy and close tsd successful, ret[0] [INFO] ME(164039,ffff87fd6440,python):2024-01-10-11:41:09.136.734 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:469] ClearDeviceContexts] Release device CPU_0 [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.136.776 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2260] ClearResPart2] End clear device context. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.136.789 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2262] ClearResPart2] Start clear AnalysisResultCacheMgr... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.136.802 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2264] ClearResPart2] End clear AnalysisResultCacheMgr. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.136.812 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2266] ClearResPart2] Start clear AnalysisContext... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.136.823 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2268] ClearResPart2] End clear AnalysisContext... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.136.833 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2270] ClearResPart2] Start clear AnalysisSchedule... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.136.972 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2272] ClearResPart2] End clear AnalysisSchedule... [INFO] DEBUG(164039,ffff87fd6440,python):2024-01-10-11:41:09.136.993 [mindspore/ccsrc/debug/debugger/debugger.cc:305] Reset] Release Debugger resource. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.137.031 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2285] ClearResPart3] Start clear ClearObjectCache... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.137.045 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2287] ClearResPart3] End clear ClearObjectCache... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.137.056 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2289] ClearResPart3] Start clear Parser... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.137.068 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2291] ClearResPart3] End clear Parser... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.137.078 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2293] ClearResPart3] Start ClearTraceStack... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.137.090 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2295] ClearResPart3] End ClearTraceStack... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.137.100 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2297] ClearResPart3] Start clear InterpretNodeRecorder... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.137.111 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2299] ClearResPart3] End clear InterpretNodeRecorder... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.137.121 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2301] ClearResPart3] Start clear parallel::entire_costgraph... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.137.153 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2303] ClearResPart3] End clear parallel::entire_costgraph... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.137.164 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2305] ClearResPart3] Start clear ProtobufLibrary... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.137.510 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2307] ClearResPart3] End clear ProtobufLibrary... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.137.525 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2309] ClearResPart3] Start clear python_adapter... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.137.537 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2311] ClearResPart3] End clear python_adapter. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.137.555 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2315] ClearSingleton] Start clear singleton... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.137.743 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2331] ClearSingleton] End clear singleton. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.137.759 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2347] ClearResAtexit] Start unload dynamic lib... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.137.781 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2349] ClearResAtexit] End unload dynamic lib... [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.515.164 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:803] DelOneNetRes] Delete one net resource start [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.521.424 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:825] DelOneNetRes] Delete one net resource end. [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.521.540 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:803] DelOneNetRes] Delete one net resource start [INFO] PIPELINE(164039,ffff87fd6440,python):2024-01-10-11:41:09.523.881 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:825] DelOneNetRes] Delete one net resource end. [WARNING] PRE_ACT(164039,ffff87fd6440,python):2024-01-10-11:41:09.986.030 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:793] DumpDynamicMemPoolStateInfo] The dynamic memory pool total allocated mem:2048M, peak used mem:5M, in used mem:0M, total idle mem:71812935M, total eager free mem:0M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [INFO] DEVICE(164039,ffff87fd6440,python):2024-01-10-11:41:09.986.106 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:136] DeInitialize] DeInitialize Ascend Memory Adapter when it is not initialize corrupted size vs. prev_size [INFO] ATRACE(164040,python):2024-01-10-11:37:02.127.368 [trace_attr.c:105](tid:164040) platform is 1. [INFO] ATRACE(164040,python):2024-01-10-11:37:02.127.591 [trace_recorder.c:114](tid:164040) use root path: /home/jenkins/ascend/atrace [INFO] ATRACE(164040,python):2024-01-10-11:37:02.127.621 [trace_signal.c:133](tid:164040) register signal handler for signo 2 succeed. [INFO] ATRACE(164040,python):2024-01-10-11:37:02.127.633 [trace_signal.c:133](tid:164040) register signal handler for signo 15 succeed. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:02.566.833 [mindspore/core/utils/ms_context.cc:225] set_backend_policy] ms set context backend policy:ge [INFO] RUNTIME(164040,python):2024-01-10-11:37:02.567.018 [runtime.cc:1159] 164040 GetAicoreNumByLevel: workingDev_=0 [INFO] RUNTIME(164040,python):2024-01-10-11:37:02.567.065 [runtime.cc:4719] 164040 GetVisibleDevices: ASCEND_RT_VISIBLE_DEVICES param was not set [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:02.567.839 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:580] SetContextSocVersion] The soc version :Ascend910A [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:02.613.960 [mindspore/ccsrc/pybind_api/ir/log_adapter_py.h:34] PyExceptionInitializer] Set exception handler [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:02.625.036 [mindspore/ccsrc/pipeline/jit/ps/init.cc:179] pybind11_init__c_expression] Start GraphExecutorPy... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:02.625.752 [mindspore/ccsrc/pipeline/jit/ps/init.cc:271] pybind11_init__c_expression] Start ParallelContext... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:02.626.386 [mindspore/ccsrc/pipeline/jit/ps/init.cc:379] pybind11_init__c_expression] Start CostModelContext... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:02.626.822 [mindspore/ccsrc/pipeline/jit/ps/init.cc:481] pybind11_init__c_expression] Start OffloadContext... [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:02.628.871 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:580] SetContextSocVersion] The soc version :Ascend910A [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:04.727.464 [mindspore/core/utils/ms_context.cc:362] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:37:04.727.548 [mindspore/ccsrc/frontend/parallel/costmodel_context.cc:30] GetInstance] Create costmodel_context [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:06.624.990 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:580] SetContextSocVersion] The soc version :Ascend910A [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:06.625.773 [mindspore/core/utils/ms_context.cc:362] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:06.625.905 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:580] SetContextSocVersion] The soc version :Ascend910A [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:06.626.428 [mindspore/core/utils/ms_context.cc:362] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] PS(164040,ffffb1c2c440,python):2024-01-10-11:37:06.626.666 [mindspore/ccsrc/ps/ps_context.cc:256] set_ms_role] MS_ROLE of this node is MS_WORKER [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:06.626.733 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:11 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:06.626.929 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:188] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281471521669360 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:06.626.953 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:13 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:06.627.043 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:188] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281471513276656 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:06.627.091 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:15 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:06.627.172 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:188] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281471504883952 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:06.627.192 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:17 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:06.627.290 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:188] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281471496491248 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:06.627.327 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:412] Connect] Can not found link destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:06.627.561 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:18 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:06.627.595 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:456] Connect] Connection 18 source: 127.0.0.1:46018, destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:06.627.619 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:475] Connect] Connected to destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:06.627.632 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:67] Connect] Connected to the tcp server 127.0.0.1:10969 successfully. [INFO] DISTRIBUTED(164040,ffff311020f0,python):2024-01-10-11:37:06.627.626 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:18 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:06.627.646 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:412] Connect] Can not found link destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:06.627.728 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:19 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:06.627.754 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:456] Connect] Connection 19 source: 127.0.0.1:46020, destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:06.627.772 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:475] Connect] Connected to destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:06.627.783 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:67] Connect] Connected to the tcp server 127.0.0.1:10969 successfully. [INFO] DISTRIBUTED(164040,ffff321040f0,python):2024-01-10-11:37:06.627.779 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:19 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:06.627.986 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:209] Register] The compute graph node: 1 has been registered successfully. [WARNING] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:06.628.063 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:195] BuildCluster] Topology build timed out., retry(1/210). [INFO] DISTRIBUTED(164040,ffff23fff0f0,python):2024-01-10-11:37:06.628.120 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:247] Heartbeat] The heartbeat thread is started. [WARNING] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:09.628.142 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:195] BuildCluster] Topology build timed out., retry(2/210). [WARNING] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:12.628.228 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:197] BuildCluster] Cluster is successfully initialized. [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:12.628.253 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:255] PostProcess] Start post processing for computing graph nodes. [WARNING] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:12.628.424 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:261] PostProcess] This node 1 rank id: 1 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:12.628.444 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:268] PostProcess] Client ip address in this cluster of this compute graph node is 127.0.0.1 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:12.628.552 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:279] PostProcess] Port range assigned for this node 1 is 9142 to 10165 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:12.628.579 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:133] node_num] Number of role MS_WORKER is 4 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:12.628.599 [mindspore/ccsrc/distributed/collective/collective_manager.cc:157] Initialize] Start initializing collective communication for backend: Ascend... [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:12.628.661 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:37] MsCollectiveCommLib] Global group name of MindSpore collective communication library is mccl_world_group [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:12.628.675 [mindspore/ccsrc/distributed/collective/collective_manager.cc:412] InitHostCommlib] Start initializing communication library on host side... [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:12.628.699 [mindspore/ccsrc/distributed/collective/collective_manager.cc:432] InitHostCommlib] Communication library on host side is successfully initialized. Global rank id: 1, global rank size: 4 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:12.628.715 [mindspore/ccsrc/distributed/collective/collective_manager.cc:470] AssignLocalRank] Host name for rank 1 is ascend85 [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:12.628.843 [mindspore/ccsrc/distributed/collective/collective_manager.cc:505] AssignLocalRank] The local rank id assigned for this process is 1. device_id of ms_context is set. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:12.640.245 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:624] SetRtDevice] Enter SetRtDevice, current initialize device number:0 [INFO] TDT(164040,python):2024-01-10-11:37:12.642.811 [process_mode_manager.cpp:109][OpenProcess][tid:164040] [ProcessModeManager] enter into open process deviceId[1] rankSize[0] [INFO] TDT(164040,python):2024-01-10-11:37:12.643.558 [process_mode_manager.cpp:379][InitTsdClient][tid:164040] [TsdClient] deviceId[1] begin to init hdc client [INFO] TDT(164040,python):2024-01-10-11:37:12.643.703 [version_verify.cpp:34][SetVersionInfo][tid:164040] VersionVerify: send client version to server [INFO] TDT(164040,python):2024-01-10-11:37:12.643.746 [version_verify.cpp:50][SetVersionInfo][tid:164040] send feature_info:{msg_type:35, features:{check before send aicpu package,}} [INFO] TDT(164040,python):2024-01-10-11:37:12.643.759 [version_verify.cpp:50][SetVersionInfo][tid:164040] send feature_info:{msg_type:37, features:{check before send open qs message,}} [INFO] TDT(164040,python):2024-01-10-11:37:12.644.181 [version_verify.cpp:66][PeerVersionCheck][tid:164040] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(164040,python):2024-01-10-11:37:12.644.196 [version_verify.cpp:87][ParseVersionInfo][tid:164040] VersionVerify: pass client version info success [INFO] TDT(164040,python):2024-01-10-11:37:12.644.206 [hdc_client.cpp:276][CheckHdcConnection][tid:164040] Service[2] create hdc success [INFO] TDT(164040,python):2024-01-10-11:37:12.644.221 [version_verify.cpp:120][SpecialFeatureCheck][tid:164040] VersionVerify: new type[35], supported [INFO] TDT(164040,python):2024-01-10-11:37:12.644.266 [process_mode_manager.cpp:748][GetDeviceCheckCode][tid:164040] [TsdClient][deviceId=1] [sessionId=1] wait package info respond [INFO] TDT(164040,python):2024-01-10-11:37:12.644.405 [process_mode_manager.cpp:379][InitTsdClient][tid:164040] [TsdClient] deviceId[1] begin to init hdc client [INFO] TDT(164040,python):2024-01-10-11:37:12.644.535 [version_verify.cpp:34][SetVersionInfo][tid:164040] VersionVerify: send client version to server [INFO] TDT(164040,python):2024-01-10-11:37:12.644.547 [version_verify.cpp:50][SetVersionInfo][tid:164040] send feature_info:{msg_type:35, features:{check before send aicpu package,}} [INFO] TDT(164040,python):2024-01-10-11:37:12.644.557 [version_verify.cpp:50][SetVersionInfo][tid:164040] send feature_info:{msg_type:37, features:{check before send open qs message,}} [INFO] TDT(164040,python):2024-01-10-11:37:12.644.705 [version_verify.cpp:66][PeerVersionCheck][tid:164040] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(164040,python):2024-01-10-11:37:12.644.717 [version_verify.cpp:87][ParseVersionInfo][tid:164040] VersionVerify: pass client version info success [INFO] TDT(164040,python):2024-01-10-11:37:12.644.725 [hdc_client.cpp:276][CheckHdcConnection][tid:164040] Service[2] create hdc success [INFO] TDT(164040,python):2024-01-10-11:37:12.644.736 [process_mode_manager.cpp:426][ConstructOpenMsg][tid:164040] [TsdClient] tsd get process sign successfully, procpid[164040] signSize[48] [INFO] TDT(164040,python):2024-01-10-11:37:12.644.748 [version_verify.cpp:112][SpecialFeatureCheck][tid:164040] VersionVerify: previous type[6], supported [INFO] TDT(164040,python):2024-01-10-11:37:12.644.768 [process_mode_manager.cpp:126][OpenProcess][tid:164040] [ProcessModeManager] deviceId[1] sessionId[1] rankSize[0], wait sub process start respond [INFO] TDT(164040,python):2024-01-10-11:37:12.864.281 [stub_process_mode_nowin.cpp:63][ProcessQueueForMdc][tid:164040] [TsdClient] it is unnecessary of current mode[0] chiptype[1] to grant queue auth to aicpusd [INFO] TDT(164040,python):2024-01-10-11:37:12.864.312 [stub_process_mode_nowin.cpp:101][OpenInHost][tid:164040] enter into OpenInHost deviceid[1] [INFO] TDT(164040,python):2024-01-10-11:37:12.864.322 [stub_process_mode_nowin.cpp:105][OpenInHost][tid:164040] host cpu not support [INFO] TDT(164040,python):2024-01-10-11:37:12.864.330 [process_mode_manager.cpp:156][OpenProcess][tid:164040] [TsdClient][deviceId=1] [sessionId=1] start hccp and computer process success [INFO] RUNTIME(164040,python):2024-01-10-11:37:12.866.998 [device.cc:340] 164040 Init: isDoubledie:0, topologytype:0 [INFO] RUNTIME(164040,python):2024-01-10-11:37:12.879.529 [npu_driver.cc:5428] 164523 GetDeviceStatus: GetDeviceStatus status=1. [INFO] ATRACE(164040,python):2024-01-10-11:37:12.879.647 [atrace_api.c:28](tid:164040) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:12.879.755 [trace_rb_log.c:84](tid:164040) [RUNTIME_ATRACE_DEV1_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:12.879.771 [atrace_api.c:32](tid:164040) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:12.879.794 [client_manager.cpp:157][SetProfilingCallback][tid:164040] [TsdClient] set profiling callback success [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:12.885.455 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:646] CreateDefaultStream] Create ascend default stream, stream id: 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:12.886.651 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:652] CreateDefaultStream] Create ascend communication stream, stream id: 1 [INFO] DEBUG(164040,ffffb1c2c440,python):2024-01-10-11:37:12.886.716 [mindspore/ccsrc/debug/debugger/debugger.cc:101] Debugger] Debugger got device_target: Ascend [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:12.886.930 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:116] Initialize] Device HBM Size:32768M, Device free HBM Size:32737M, Reserved HBM size for Other Components(HCCL/rts/etc.):2057M, Recommend Reserved HBM size for Other Components:2046M, User define MindSpore HBM Size:0G, MindSpore Used HBM Size:30680M. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:13.040.796 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:301] MallocFromRts] Call rtMalloc to allocate device memory Success, size: 32170311680 bytes, address start: 0x124100000000 end: 0x12487d800000 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:13.040.833 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:129] Initialize] Ascend Memory Adapter initialize success, Memory Statistics: Device HBM memory size: 32768M MindSpore Used memory size: 30680M MindSpore memory base address: 0x124100000000 Total Static Memory size: 0M Total Dynamic memory size: 0M Dynamic memory size of this graph: 0M [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:13.042.082 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:463] SetDisableReuseMemoryFlag] DISABLE_REUSE_MEMORY is not set in ENV. Now set to default value 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:13.042.109 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:500] SetHcclOptions] No hccl mode. If use hccl, make sure [RANK_TABLE_FILE,RANK_ID,DEVICE_ID] all be set in ENV. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:13.042.160 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:366] GetGeOptions] JOB_ID is not set in ENV. Now set to default value 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:13.042.178 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:384] GetGeOptions] Set proto lib path failed! [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:13.042.190 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:304] SetAscendConfig] GE topo sorting mode is: [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:13.042.204 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:316] SetAscendConfig] Set GE topo mode to memory-priority. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:13.042.220 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:321] SetAscendConfig] Set staticMemoryPolicy to default mode. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:13.042.232 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:329] SetAscendConfig] The default value of jit_compile is set to 2. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:13.042.244 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:285] SetAscendHF32Config] The default value of allow_matmul_hf32 and allow_conv_hf32 are set by CANN. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:13.042.255 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:294] SetAscendHF32Config] allow_matmul_hf32: , allow_conv_hf32: [TRACE] GE(164040,python):2024-01-10-11:37:13.042.307 [status:INIT] [ge_api.cc:144]164040 GEInitializeImpl:GEInitialize start [INFO] PROFILING(164040,python):2024-01-10-11:37:13.270.399 [msprofiler_impl.cpp:156] >>> (tid:164040) ProfNotifySetDevice called, is open: 1, devId: 1 [INFO] PROFILING(164040,python):2024-01-10-11:37:13.270.565 [platform.cpp:38] >>> (tid:164040) Profiling platform version: 1.0. [INFO] PROFILING(164040,python):2024-01-10-11:37:13.270.583 [ai_drv_dev_api.cpp:384] >>> (tid:164040) Succeeded to DrvGetApiVersion version: 0x72313 [TRACE] GE(164040,python):2024-01-10-11:37:13.322.623 [status:RUNNING] [ge_api.cc:211]164040 GEInitializeImpl:Initializing environment [INFO] GE(164040,python):2024-01-10-11:37:13.322.682 [gelib.cc:98][EVENT]164040 Initialize:[GEPERFTRACE] GE Init Start [INFO] GE(164040,python):2024-01-10-11:37:13.323.025 [gelib.cc:307][EVENT]164040 SystemInitialize:Online infer init GELib success, device id :1 [INFO] DVPP(164040,python):2024-01-10-11:37:13.708.191 [dvpp_engine.cc:41][ENGINE][Initialize:41][tid:164040]dvpp engine do not support [INFO] TUNE(164040,python):2024-01-10-11:37:13.713.173 [cann_kb_pyfunc_mgr.cpp:72][CANNKB][Tid:164040]"CannKbPyfuncMgr: Enter PyObjectInit, reference_ is 0!" [INFO] TUNE(164040,python):2024-01-10-11:37:13.713.217 [handle_manager.cpp:115][CANNKB][Tid:164040]"Start to run init functions to load dynamic python lib!" [INFO] TUNE(164040,python):2024-01-10-11:37:13.713.277 [handle_manager.cpp:407][CANNKB][Tid:164040]"Init functions of loading dynamic python lib end!" [INFO] TUNE(164040,python):2024-01-10-11:37:13.713.288 [cann_kb_pyfunc_mgr.cpp:24][CANNKB][Tid:164040]"CANN_KB_Py has already been initialized." [INFO] TUNE(164040,python):2024-01-10-11:37:13.713.379 [cann_kb_pyfunc_mgr.cpp:117][CANNKB][Tid:164040]"CannKbPyfuncMgr: Run PyObjectInit successfully!" [INFO] HCCL(164040,python):2024-01-10-11:37:25.462.855 [plugin_manager.cc:42][164040]hcom running normal mode. [INFO] DVPP(164040,python):2024-01-10-11:37:25.463.546 [dvpp_engine.cc:92][ENGINE][GetOpsKernelInfoStores:92][tid:164040]dvpp ops kernel info store do not support [INFO] DVPP(164040,python):2024-01-10-11:37:25.463.710 [dvpp_engine.cc:69][ENGINE][GetGraphOptimizerObjs:69][tid:164040]dvpp graph optimizer do not support [INFO] DVPP(164040,python):2024-01-10-11:37:26.125.607 [dvpp_ops_kernel_builder.cc:48][ENGINE][Initialize:48][tid:164040]dvpp ops kernel builder do not support [INFO] GE(164040,python):2024-01-10-11:37:26.133.864 [gelib.cc:169][EVENT]164040 Initialize:[GEPERFTRACE] The time cost of GELib::Initialize is [12811101] micro second. [TRACE] GE(164040,python):2024-01-10-11:37:26.223.808 [status:STOP] [ge_api.cc:255]164040 GEInitializeImpl:GEInitialize finished [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:26.224.047 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_res_manager.cc:168] GeSetContextOptions] Set GE atomic clean policy to 1. [TRACE] GE(164040,python):2024-01-10-11:37:26.224.121 [status:INIT] [ge_api.cc:398]164040 Session:Start to construct session. [TRACE] GE(164040,python):2024-01-10-11:37:26.224.140 [status:RUNNING] [ge_api.cc:408]164040 Session:Creating session [INFO] GE(164040,python):2024-01-10-11:37:26.224.653 [graph_var_manager.cc:1445][EVENT]164040 SetMemoryMallocSize:Total memory size is 34359738368 [INFO] GE(164040,python):2024-01-10-11:37:26.224.674 [graph_var_manager.cc:1424][EVENT]164040 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] PROFILING(164040,python):2024-01-10-11:37:26.225.036 [msprofiler_impl.cpp:156] >>> (tid:164040) ProfNotifySetDevice called, is open: 1, devId: 1 [TRACE] GE(164040,python):2024-01-10-11:37:26.225.873 [status:RUNNING] [ge_api.cc:411]164040 Session:Session id is 0 [TRACE] GE(164040,python):2024-01-10-11:37:26.225.898 [status:STOP] [ge_api.cc:420]164040 Session:Session Constructor finished [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:26.225.930 [mindspore/ccsrc/transform/graph_ir/graph_runner.cc:53] NewSession] Create new GE session success! [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:26.225.953 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:210] SetGeSession] Add a new Ge Session success [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:26.225.995 [mindspore/ccsrc/transform/graph_ir/graph_runner.cc:65] GraphRunner] ME run in ONE_DEVICE strategy mode [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:26.226.038 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:246] SetGraphRunner] Add a new GraphRunner success [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:26.226.080 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:238] InitGe] Create session and graphrunner successful. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:26.226.097 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:242] InitGe] Init ge successful, ge reference = 1. [INFO] PROFILING(164040,python):2024-01-10-11:37:26.236.233 [platform.cpp:38] >>> (tid:164040) Profiling platform version: 1.0. [INFO] PROFILING(164040,python):2024-01-10-11:37:26.236.264 [ai_drv_dev_api.cpp:384] >>> (tid:164040) Succeeded to DrvGetApiVersion version: 0x72313 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:26.236.446 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:193] Initialize] Call aclInit successfully. [TRACE] GE(164040,python):2024-01-10-11:37:26.236.548 [status:INIT] [ge_api.cc:144]164040 GEInitializeImpl:GEInitialize start [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:26.237.815 [mindspore/ccsrc/distributed/collective/collective_manager.cc:455] InitDeviceCommLib] Start initializing communication library on device side... [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:26.237.884 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:291] OpenTsd] Device id = 1, rank size = 4. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:26.246.390 [mindspore/ccsrc/plugin/device/ascend/hal/device/tensorprint_utils.cc:405] CreateChannel] For Print ops, select MBUF channel. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:26.246.425 [mindspore/ccsrc/plugin/device/ascend/hal/device/tensorprint_utils.cc:420] CreateTensorPrintThread] Success to create acl channel handle, tsd reference = 1. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:26.247.058 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:103] MbufDataHandler] Channel ms_tensor_dump begins the construction process. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:26.247.486 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:103] MbufDataHandler] Channel ms_tensor_summary begins the construction process. [INFO] DEVICE(164040,fffee95850f0,python):2024-01-10-11:37:26.247.524 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:161] HandleData] Channel ms_tensor_dump starts executing HandleData. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:26.247.927 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:103] MbufDataHandler] Channel ms_image_summary begins the construction process. [INFO] DEVICE(164040,fffee8d840f0,python):2024-01-10-11:37:26.247.996 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:161] HandleData] Channel ms_tensor_summary starts executing HandleData. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:26.248.399 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:103] MbufDataHandler] Channel ms_scalar_summary begins the construction process. [INFO] DEVICE(164040,fffedbfff0f0,python):2024-01-10-11:37:26.248.445 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:161] HandleData] Channel ms_image_summary starts executing HandleData. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:26.248.735 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:103] MbufDataHandler] Channel ms_histogram_summary begins the construction process. [INFO] DEVICE(164040,fffedb7fe0f0,python):2024-01-10-11:37:26.248.783 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:161] HandleData] Channel ms_scalar_summary starts executing HandleData. [INFO] HCCL_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:26.249.046 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:202] InitHccl] Start init hccl adapter. [INFO] DEVICE(164040,fffedaffd0f0,python):2024-01-10-11:37:26.249.081 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:161] HandleData] Channel ms_histogram_summary starts executing HandleData. [INFO] HCCL_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:26.249.190 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:341] InitKernelInfoStore] Start init hccl kernel info store. [INFO] HCCL_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:26.249.256 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:358] InitKernelInfoStore] Get builder ops_kernel_info_hccl [INFO] HCCL(164040,python):2024-01-10-11:37:26.249.283 [plugin_manager.cc:42][164040]hcom running normal mode. [INFO] HCCL_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:26.249.358 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:380] InitKernelInfoStore] Init hccl kernel info store success. [INFO] HCCL_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:26.249.376 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:528] InitHcclExec] Start init hccl exec. [INFO] HCCL(164040,python):2024-01-10-11:37:26.252.089 [hcom_executor.cc:32][164040][Initialize][HcomExecutor]Hcom Excutor Initialize end. ret[0] [INFO] HCCL_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:26.252.136 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:534] InitHcclExec] Hcom DynamicKernel Initialize success [INFO] HCCL_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:26.252.158 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:540] InitHcclExec] InitHcclExec success [INFO] HCCL_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:26.252.170 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:220] InitHccl] Init hccl adapter success. [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:26.252.187 [mindspore/ccsrc/distributed/collective/collective_manager.cc:458] InitDeviceCommLib] Communication library on device side is successfully initialized. [WARNING] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:26.252.209 [mindspore/ccsrc/distributed/collective/collective_manager.cc:220] CreateCommunicationGroup] Start to create communication group: hccl_world_group [const vector]{0, 1, 2, 3} [WARNING] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:26.252.537 [mindspore/ccsrc/distributed/collective/collective_manager.cc:278] CreateCommunicationGroup] Begin initialize communication group on the device side: hccl_world_group [INFO] HCCL(164040,python):2024-01-10-11:37:26.252.694 [externalinput.cc:310][166232]environmental variable HCCL_CONNECT_TIMEOUT is set, timeOut[600] [INFO] HCCL(164040,python):2024-01-10-11:37:26.252.739 [externalinput.cc:598][166232]environmental variable HCCL_IF_IP is not set [INFO] HCCL(164040,python):2024-01-10-11:37:26.252.754 [externalinput.cc:655][166232]environmental variable HCCL_SOCKET_IFNAME is not set, default[EmptyString] [INFO] HCCL(164040,python):2024-01-10-11:37:26.252.769 [externalinput.cc:582][166232]environmental variable HCCL_IF_BASE_PORT is not set [INFO] HCCL(164040,python):2024-01-10-11:37:26.252.858 [externalinput.cc:282][166232]environmental variable HCCL_HIGH_PERF_ENABLE is not set [INFO] HCCL(164040,python):2024-01-10-11:37:26.252.889 [op_base.cc:405][166232]Entry-HcclCommInitRootInfo:ranks[4], rank[1], rootinfo: host ip[8.92.9.85] port[60000] nicDeploy[1] identifier[8.92.9.85%enp189s0f0_60000_0_1704857845691656], deviceLogicId[1] [INFO] HCCP(164040,python):2024-01-10-11:37:26.253.254 [ra_host.c:1882]tid:166232,ra_get_ifnum(1882) : Input parameters: phy_id[0], nic_position:[0] [INFO] HCCL(164040,python):2024-01-10-11:37:26.255.431 [adapter_hccp.cc:821][166232][Get][HostIf]hrtGetIfNum success. ifAddrNum[7]. [INFO] HCCP(164040,python):2024-01-10-11:37:26.255.456 [ra_host.c:1930]tid:166232,ra_get_ifaddrs(1930) : Input parameters: phy_id[0], nic_position:[0], interface num[7] [INFO] HCCL(164040,python):2024-01-10-11:37:26.256.626 [sal.cc:383][166232]nic class[normal]: find nic[8.92.9.85%enp189s0f0] success. [INFO] HCCP(164040,python):2024-01-10-11:37:26.256.691 [ra_host.c:1722]tid:166232,ra_socket_set_white_list_status(1722) : Input parameters: enable[0] [INFO] HCCP(164040,python):2024-01-10-11:37:26.256.711 [ra_host.c:293]tid:166232,ra_init(293) : Input parameters: phy_id[1], nic_position:[0] [INFO] HCCP(164040,python):2024-01-10-11:37:26.257.047 [rs_ssl.c:1104]tid:166232,rs_ssl_init(1104) : TLS SWITCH (0) [INFO] HCCP(164040,python):2024-01-10-11:37:26.257.192 [rs_epoll.c:470]tid:166233,rs_epoll_handle(470) : pthread[epoll_pthread] is alive! [INFO] HCCP(164040,python):2024-01-10-11:37:26.257.220 [rs_epoll.c:595]tid:166234,rs_connect_handle(595) : pthread[connect_pthread] is alive! [INFO] HCCP(164040,python):2024-01-10-11:37:26.257.243 [rs.c:403]tid:166232,rs_init(403) : rs init success, chip_id[1] [INFO] TDT(164040,python):2024-01-10-11:37:26.257.290 [process_mode_manager.cpp:109][OpenProcess][tid:166232] [ProcessModeManager] enter into open process deviceId[1] rankSize[2] [INFO] TDT(164040,python):2024-01-10-11:37:26.257.626 [process_mode_manager.cpp:705][GetDeviceCheckCode][tid:166232] [ProcessModeManager][deviceId=1] aicpu package already exist in device [INFO] TDT(164040,python):2024-01-10-11:37:26.257.668 [process_mode_manager.cpp:426][ConstructOpenMsg][tid:166232] [TsdClient] tsd get process sign successfully, procpid[164040] signSize[48] [INFO] TDT(164040,python):2024-01-10-11:37:26.257.795 [process_mode_manager.cpp:126][OpenProcess][tid:166232] [ProcessModeManager] deviceId[1] sessionId[1] rankSize[2], wait sub process start respond [INFO] TDT(164040,python):2024-01-10-11:37:26.424.022 [stub_process_mode_nowin.cpp:63][ProcessQueueForMdc][tid:166232] [TsdClient] it is unnecessary of current mode[0] chiptype[1] to grant queue auth to aicpusd [INFO] TDT(164040,python):2024-01-10-11:37:26.424.044 [stub_process_mode_nowin.cpp:101][OpenInHost][tid:166232] enter into OpenInHost deviceid[1] [INFO] TDT(164040,python):2024-01-10-11:37:26.424.054 [stub_process_mode_nowin.cpp:105][OpenInHost][tid:166232] host cpu not support [INFO] TDT(164040,python):2024-01-10-11:37:26.424.063 [process_mode_manager.cpp:156][OpenProcess][tid:166232] [TsdClient][deviceId=1] [sessionId=1] start hccp and computer process success [INFO] HCCP(164040,python):2024-01-10-11:37:26.424.075 [ra_host.c:293]tid:166232,ra_init(293) : Input parameters: phy_id[1], nic_position:[1] [INFO] HCCP(164040,python):2024-01-10-11:37:26.424.102 [ra_hdc.c:1465]tid:166232,ra_hdc_init(1465) : hdc init start! logic id is 1, phy id is 1 [INFO] HCCP(164040,python):2024-01-10-11:37:26.424.385 [ra_hdc.c:1500]tid:166232,ra_hdc_init(1500) : hdc init OK! phy_id[1] [INFO] HCCP(164040,python):2024-01-10-11:37:26.426.736 [ra_host.c:454]tid:166232,ra_socket_init_v1(454) : socket init:mode=0 phy_id=1 family=2 ip=8.92.9.85 [INFO] HCCL(164040,python):2024-01-10-11:37:26.426.771 [adapter_hccp.cc:988][166232][Get][DeviceIP]hrtGetIfNum success. ifAddrNum[2]. [INFO] HCCP(164040,python):2024-01-10-11:37:26.426.780 [ra_host.c:1930]tid:166232,ra_get_ifaddrs(1930) : Input parameters: phy_id[1], nic_position:[1], interface num[2] [INFO] HCCL(164040,python):2024-01-10-11:37:26.431.387 [adapter_hccp.cc:1018][166232]hrtGetIfAddress: idx[0] ifname[eth1] ip[192.168.101.101%eth1] [INFO] HCCL(164040,python):2024-01-10-11:37:26.431.401 [topoinfo_detect.cc:472][166232]select AF_INET family as device socket family. [INFO] HCCP(164040,python):2024-01-10-11:37:26.431.445 [ra_host.c:825]tid:166232,ra_socket_batch_connect(825) : Input parameters: [0]th, phy_id[1], local_ip[8.92.9.85], remote_ip[8.92.9.85], tag:[topo_detect_default_tag_60000] [INFO] HCCP(164040,python):2024-01-10-11:37:26.694.051 [ra_host.c:863]tid:166232,ra_socket_batch_close(863) : Input parameters: [0]th, phy_id[1], local_ip[8.92.9.85] [INFO] HCCP(164040,python):2024-01-10-11:37:26.737.906 [ra_host.c:525]tid:166232,ra_socket_deinit(525) : Input parameters: phy_id[1] family[2] local_ip[8.92.9.85] [INFO] HCCP(164040,python):2024-01-10-11:37:26.737.938 [rs.c:1257]tid:166232,rs_socket_deinit(1257) : socket deinit success, phy_id:1, local_ip:8.92.9.85 [INFO] HCCP(164040,python):2024-01-10-11:37:26.737.955 [ra_host.c:349]tid:166232,ra_deinit(349) : Input parameters: phy_id[1], nic_position:[0] [INFO] HCCP(164040,python):2024-01-10-11:37:26.858.333 [rs.c:1460]tid:166232,rs_deinit(1460) : rs_deinit chip_id[1] ok [INFO] HCCP(164040,python):2024-01-10-11:37:26.858.354 [ra_host.c:349]tid:166232,ra_deinit(349) : Input parameters: phy_id[1], nic_position:[1] [INFO] HCCP(164040,python):2024-01-10-11:37:26.858.364 [ra_hdc.c:1535]tid:166232,ra_hdc_deinit(1535) : hdc deinit start! phy_id[1] [INFO] HCCP(164040,python):2024-01-10-11:37:26.858.500 [ra_hdc.c:1570]tid:166232,ra_hdc_deinit(1570) : hdc deinit OK! phy_id[1] [INFO] ATRACE(164040,python):2024-01-10-11:37:26.858.714 [atrace_api.c:28](tid:166232) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:26.858.772 [trace_rb_log.c:84](tid:166232) [HCCL_166232_1] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:26.858.785 [atrace_api.c:32](tid:166232) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:26.859.193 [process_mode_manager.cpp:109][OpenProcess][tid:166232] [ProcessModeManager] enter into open process deviceId[1] rankSize[2] [INFO] HCCP(164040,python):2024-01-10-11:37:26.859.212 [ra_host.c:293]tid:166232,ra_init(293) : Input parameters: phy_id[1], nic_position:[1] [INFO] HCCP(164040,python):2024-01-10-11:37:26.859.289 [ra_hdc.c:1465]tid:166232,ra_hdc_init(1465) : hdc init start! logic id is 1, phy id is 1 [INFO] HCCP(164040,python):2024-01-10-11:37:26.859.556 [ra_hdc.c:1500]tid:166232,ra_hdc_init(1500) : hdc init OK! phy_id[1] [INFO] HCCP(164040,python):2024-01-10-11:37:26.861.865 [ra_host.c:389]tid:166232,ra_socket_init(389) : socket init:mode=1 phy_id=1 family=2 ip=1.0.0.0 [INFO] HCCP(164040,python):2024-01-10-11:37:26.862.522 [ra_host.c:903]tid:166232,ra_socket_listen_start(903) : Input parameters: [0]th, phy_id[1], local_ip[1.0.0.0] [INFO] HCCL(164040,python):2024-01-10-11:37:26.865.164 [hccl_impl.cc:430][166232]hccl algorithm: [Module(aiserver)] there are 4 device in level0, using fullmesh algo [TRACE] HCCL(164040,python):2024-01-10-11:37:26.869.776 [status:init] [op_base.cc:481][166232]HcclCommInitRootInfo success,take time [617095]us, rankNum[4], rank[1],rootInfo identifier[8.92.9.85%enp189s0f0_60000_0_1704857845691656], server[8.92.9.85%enp189s0f0], device[1] [WARNING] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:26.869.829 [mindspore/ccsrc/distributed/collective/collective_manager.cc:287] CreateCommunicationGroup] End initialize communication group on the device side: hccl_world_group [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:37:26.869.851 [mindspore/ccsrc/distributed/collective/collective_manager.cc:182] Initialize] End initializing collective communication for backend: Ascend [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:26.869.881 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:285] RecordInitStatus] Status record: system init. [WARNING] ME(164040:281473664074816,MainProcess):2024-01-10-11:37:28.589.091 [mindspore/parallel/_utils.py:259] You are suggested to use mindspore.context.set_auto_parallel_context(parameter_broadcast=True) or mindspore.common.set_seed() to share parameters among multi-devices. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:29.356.965 [mindspore/ccsrc/minddata/dataset/util/task_manager.cc:161] DoServiceStart] Starting Task Manager. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:29.357.743 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at train-labels-idx1-ubyte. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:29.357.780 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at train-images-idx3-ubyte. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:29.357.903 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:29.359.650 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:29.359.777 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:29.359.801 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:29.359.818 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:29.359.849 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:29.359.916 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:29.359.942 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:29.359.956 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:29.359.975 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:29.359.993 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:29.360.027 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:29.360.041 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:29.360.061 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:29.360.089 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:29.360.382 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at train-labels-idx1-ubyte. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:29.360.406 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at train-images-idx3-ubyte. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.015.271 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.015.396 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.061.054 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.061.197 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.061.220 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.061.237 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.061.268 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.061.334 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.061.362 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.061.378 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.061.400 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.061.421 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.061.453 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.061.467 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.061.513 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.061.542 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.061.900 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at train-labels-idx1-ubyte. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.061.929 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at train-images-idx3-ubyte. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.654.737 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.654.870 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.700.554 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.700.698 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.700.720 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.700.738 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.700.768 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.700.838 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.700.865 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.700.880 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.700.925 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.700.944 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.700.977 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.700.991 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.701.011 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.701.042 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.701.343 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] ME(164040:281473664074816,MainProcess):2024-01-10-11:37:30.759.621 [mindspore/dataset/engine/datasets.py:4269] queue_name is newly generated. value is 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.762.405 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.762.539 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.762.565 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.762.582 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.762.636 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.762.703 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.762.730 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.762.746 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.762.765 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.762.784 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.762.817 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.762.831 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.762.851 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.762.880 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:30.763.193 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:30.763.387 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1729] InitExecDatasetVm] Start InitDataSet Entry [INFO] DEBUG(164040,ffffb1c2c440,python):2024-01-10-11:37:30.763.704 [mindspore/ccsrc/common/debug/env_config_parser.cc:152] ParseFromFile] The 'env_config_path' in 'mindspore.context.set_context(env_config_path={path})' is empty. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:37:30.763.737 [mindspore/ccsrc/backend/graph_compiler/transform.cc:573] CreateBackend] CreateBackend is: ge [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:30.763.756 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:30.763.770 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEBUG(164040,ffffb1c2c440,python):2024-01-10-11:37:30.763.930 [mindspore/ccsrc/debug/debugger/debugger.cc:129] Init] Debugger got device_id: 1 [INFO] DEBUG(164040,ffffb1c2c440,python):2024-01-10-11:37:30.763.956 [mindspore/ccsrc/debug/debugger/debugger.cc:131] Init] Debugger got device_target: Ascend [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:30.863.733 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:433] Initialize] The actor thread number: 5, the kernel thread number: 25 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:30.864.054 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:30.864.086 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:30.864.101 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1252] SetRunMode] Run graph mode with kernel by kernel by configuration. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:37:30.864.628 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:546] CompileGraphs] Status record: start compile function graph: _anonymous__1 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.864.758 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unify_mindir_pm_0_erase_invalid_micro_depend in 11.17 us [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:37:30.865.459 [mindspore/ccsrc/utils/anfalgo.cc:1736] IsNodeOutputDynamicShape] Invalid base shape, node: Default/Return-op0 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:30.865.558 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:30.865.577 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:30.865.621 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:30.865.634 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:30.865.657 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:30.865.671 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:37:30.865.751 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:727] CompileGraph] Compile graph: _anonymous__1, Split segments size: 2 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:37:30.865.808 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:762] CompileGraphFromSegment] Compile normal segment, the first node: @_anonymous__1:CNode_2{[0]: ValueNode InitDataSetQueue} [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:30.865.896 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:421] CompileGraph] Status record: start compile graph. [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:30.865.958 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:1940] ConstructKernelGraph] Create graph: 0 [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:30.866.173 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:2697] ConstructOutput] Output:@_anonymous__1:CNode_2{[0]: ValueNode InitDataSetQueue} [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.866.658 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:193] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:0 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.866.735 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 9.52 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.866.866 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 78.74 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.011 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:154] CommonUnifyMindIR] start common unify mindir opt graph:0 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.061 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.170 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 103.28 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.190 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.226 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 31.42 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.243 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.258 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 14.25 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.271 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.317 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 43 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.431 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:79] BackendCommonOptimization] Status record: start common optimization. graph id: 0 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.473 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: add_dynamic_shape_attr [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.531 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_0_add_dynamic_shape_attr in 53.5 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.548 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.611 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_1_convert_dynamic_broadcast_to in 57.54 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.661 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_2_convert_const_input_to_attr in 30.11 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.696 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_3_custom_op_const_input_to_attr in 15.28 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.729 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 14.87 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.808 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 59.38 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.828 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 1.19 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.888 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_7_flatten_concat_fission in 32.82 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.934 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 25.61 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.867.974 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_9_broadcast_to_fusion in 22.05 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.239 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 240.7 us [WARNING] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.349 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:92] BackendCommonOptimization] [PROF]backend_common_optimization costs 917 usec. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.367 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:99] BackendCommonOptimization] Status record: end common optimization. graph id: 0 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.561 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_0_renorm_split in 44.09 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.583 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.668 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_1_reduce_axis_update in 78.7 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.690 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim HistogramFixedWidth not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.706 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_2_histogram_fixed_width_fusion in 15.45 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.737 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim ClipByNorm not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.755 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_3_clip_by_norm_fission in 32.24 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.787 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.802 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_4_tensor_arry_add_flow_cond1 in 31.35 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.816 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayGather not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.830 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_5_tensor_arry_add_flow_cond2 in 11.75 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.842 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.856 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_6_ge_tensor_arry_cast_index in 11.39 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.873 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArray not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.895 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_7_tensor_array_prepare in 24.66 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.908 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.960 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_8_space_to_batch_nd_attr_update in 46.98 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.868.976 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.005 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_9_batch_to_space_nd_attr_update in 25.84 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.020 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: avg_pool_grad_unify_mindir [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.079 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_10_avg_pool_grad_unify_mindir in 54.48 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.141 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_11_adam_weight_decay_unify_mindir in 39.78 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.201 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_12_cdist_fission in 36.43 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.246 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_13_cdist_grad_fission in 21.6 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.329 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_14_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 61.85 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.388 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_15_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 35.76 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.430 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_16_sparse_softmax_cross_entropy_with_logits_unify_mindir in 18.14 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.468 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_17_dropout_unify_mindir1 in 16.4 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.487 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.536 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_18_dropoutgrad_unify_mindir in 45.75 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.571 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.591 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_19_neighbor_exchange_unify_mindir in 33.27 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.608 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.622 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_20_neighbor_exchange_v2_unify_mindir in 12.43 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.647 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.661 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_21_neighbor_exchange_v2_grad_unify_mindir in 12.76 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.750 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.770 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_22_all_to_all_unify_mindir in 93.66 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.786 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim QuantDTypeCast not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.800 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_23_quant_dtype_cst_adjust in 14.68 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.815 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim FSEDecode not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.829 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_24_fse_decode_adjust in 13.11 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.856 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.871 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_25_bn_split in 27.09 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.884 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.950 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_26_bn_grad_unify_mindir in 61.97 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.967 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.981 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_27_bn_grad_split in 13.25 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.869.994 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.870.007 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_28_batchnorm_to_bninfer in 12.05 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.870.021 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.870.034 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_29_batchnormgrad_to_bninfergrad in 11.89 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.870.047 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.870.060 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_30_batch_norm_grad_infer_fission in 11.62 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.870.139 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_31_ascend_mindir_op_adapter in 61.59 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.870.177 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_32_DropoutGenMask in 1.27 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.870.230 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_33_lamb_fission_ge in 33.95 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.870.285 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_34_print_insert_placeholder_for_tensor_name in 35.82 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.870.341 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_35_getnext_for_ge in 35.5 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.870.399 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_36_sync_bn_split in 34.96 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.870.442 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_37_sync_bn_grad_split in 18.91 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.870.493 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_38_adaptive_max_pool2d_ge_fusion in 32.28 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.870.530 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_39_avg_pool_grad_for_ge in 17 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.870.549 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:260] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.870.661 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_40_FlashAttentionFusionV1 in 109.41 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.870.680 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:374] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.870.789 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_41_FlashAttentionFusionV1 in 106.93 us [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:30.871.122 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:30.871.148 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:30.871.162 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:132] GetRunMode] RunMode::kKernelMode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.871.343 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 140.03 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.871.549 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_0_seed_adapter in 69.65 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.871.574 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: env_op_attr_update [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.871.641 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_1_env_op_attr_update in 60.91 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.871.699 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_2_process call inline in 25.69 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.871.756 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_3_expander_fallback in 34.24 us [WARNING] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.871.837 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:139] GEBackendOptimizeACL] [PROF]ascend_backend_optimize_acl costs 373 usec. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:30.871.870 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] InitDataSetQueue is not defined in opdef. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.872.138 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 22.26 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.872.220 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 49.23 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.872.282 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 36.46 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.872.356 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 50.42 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.872.398 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_4_insert_type_transform_op in 19.03 us [WARNING] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.872.482 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:179] GEBackendOptimizeACLAfterKernelSelect] [PROF]ascend_backend_optimize_acl_after_kernel_select costs 448 usec. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.872.539 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_0_DropoutGenMask in 0.6 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.872.594 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_1_cse in 33.38 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.872.652 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_redundant_op in 37.05 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.872.683 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_3_eliminate_maketuple_getitem in 9.88 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.872.701 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_4_MergeTransData in 0.9 us [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:30.872.821 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive InitDataSetQueue [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:30.872.935 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive InitDataSetQueue [WARNING] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:30.872.955 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:262] CreateKernel] [PROF]create_kernel costs 142 usec. [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.008 [mindspore/ccsrc/backend/common/session/session_basic.cc:1140] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 0 start [INFO] DEBUG(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.033 [mindspore/ccsrc/debug/summary/summary.cc:33] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 0 start [INFO] DEBUG(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.049 [mindspore/ccsrc/debug/summary/summary.cc:54] RecurseSetSummaryNodesForAllGraphs] The total summary nodes is: 0 for graph: 0 [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.143 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:891] PrintGraphExecuteOrder] Graph 0 execution order: [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.189 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[0], node name[Default/InitDataSetQueue-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_0:CNode_2{[0]: ValueNode InitDataSetQueue}] [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.222 [mindspore/ccsrc/backend/common/somas/somas.cc:143] Assign] Start Somas Assign for graph 0 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.254 [mindspore/ccsrc/backend/common/somas/somas.cc:151] Assign] Somas Configure success, configuration info: Device Name: Ascend Run by execution order: 1 Enable debug log: 0 Debug log path: . [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.270 [mindspore/ccsrc/backend/common/somas/somas.cc:154] Assign] Start Initialize SOMAS Model [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.335 [mindspore/ccsrc/backend/common/somas/somas.cc:1065] GraphOutputProcess] Set 0 graph output tensors' aligned size to 0. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.376 [mindspore/ccsrc/backend/common/somas/somas.cc:1135] RefNodeProcess] Special Tensor total size: RefNode: input 0 output 0 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.398 [mindspore/ccsrc/backend/common/somas/somas.cc:551] InitSomasModel] No Tensor from graph 0 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.409 [mindspore/ccsrc/backend/common/somas/somas.cc:157] Assign] End Initialize SOMAS Model [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.420 [mindspore/ccsrc/backend/common/somas/somas.cc:160] Assign] No Somas Tensor in graph 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.431 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:310] DoSomas] Somas allocate success for graph 0 somas size: 0 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.457 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:801] CreateDeviceAddress] Status record: start create device address. graph id: 0 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.529 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:808] CreateDeviceAddress] Status record: end create device address. graph id: 0 [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.567 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1129] CacheGraphOutputToFrontNodeWithIndex] Get graph backend output nodes. [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.588 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1137] CacheGraphOutputToFrontNodeWithIndex] Get graph front output nodes. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.606 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:507] CompileGraph] Status record: end compile graph. graph id: 0 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.684 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:805] CompileGraphFromSegment] Compile cut segment, the cut node: @_anonymous__1:CNode_3{[0]: ValueNode Return, [1]: CNode_2} [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.826 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:521] Transform] Graph(kernel_graph_0) transforms actor begin, strategy:pipeline_with_execution_order [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.942 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1232] BuildLoopCountActor] Create loop count actor: kernel_graph_0_LoopCountActor [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.962 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1265] BuildOutputActor] Create output actor: kernel_graph_0_OutputActor [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:30.873.979 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1285] BuildDataPrepareActor] Create data prepare actor: kernel_graph_0_DataPrepareActor [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:30.874.010 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1518] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_0 start. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:30.874.029 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1520] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_0 end. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:30.874.122 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 1_actor_set_kernel_graph_0_memory_actor_insert in 1.23 us [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:30.874.142 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 2_actor_set_kernel_graph_0_invalid_data_arrow_elimination in 1.07 us [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:30.874.183 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 3_actor_set_kernel_graph_0_multi_actor_fusion in 24.06 us [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:30.874.200 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 4_actor_set_kernel_graph_0_batch_data_arrow_fusion in 0.879998 us [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:30.874.218 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:554] Transform] Graph(kernel_graph_0) transforms actor end. [WARNING] VM(164040,ffffb1c2c440,python):2024-01-10-11:37:30.874.269 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:612] CompileGraphs] [PROF]compile_func_graph costs 9614 usec. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:37:30.874.291 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:615] CompileGraphs] Status record: end compile function graph: _anonymous__1, produce actor: kernel_graph_0 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:37:30.874.313 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_0 [INFO] GE(164040,python):2024-01-10-11:37:30.874.853 [scalable_config.cc:55][EVENT]167129 ScalableConfig:device total max size: 34359738368, page_mem_size_total_thresold: 32641751449, uncacheable_size_threshold: 17179869184 [INFO] GE(164040,python):2024-01-10-11:37:30.957.818 [graph_var_manager.cc:1424][EVENT]167129 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:30.957.946 [graph_manager.cc:1248][EVENT]167129 PreRun:PreRun start: graph node size 1, session id 1, graph id 0, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:30.958.863 [atrace_api.c:28](tid:167129) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:30.958.943 [trace_rb_log.c:84](tid:167129) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:30.958.959 [atrace_api.c:32](tid:167129) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:30.958.991 [client_manager.cpp:157][SetProfilingCallback][tid:167129] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:30.960.057 [parallel_partitioner.cc:165][EVENT]167129 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.960.125 [parallel_partitioner.cc:178][EVENT]167129 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.960.187 [graph_prepare.cc:1378][EVENT]167129 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.960.865 [graph_manager.cc:1050][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [700] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.960.894 [graph_manager.cc:1052][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.960.975 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [1] [INFO] GE(164040,python):2024-01-10-11:37:30.961.005 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.961.132 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [115] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.961.146 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.961.272 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [44] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.961.285 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.961.301 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.961.423 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.961.443 [graph_manager.cc:1054][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [536] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.969.178 [graph_manager.cc:1055][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [7719] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.969.949 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:30.969.979 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.969.990 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [24] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.969.999 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [148] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.970.008 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [33] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.970.017 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:30.970.026 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.970.034 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.970.052 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [11] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.160 [graph_manager.cc:1056][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [1943] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.971.217 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.234 [graph_prepare.cc:1982][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [41] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.971.389 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:30.971.407 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.417 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.426 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [49] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.434 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [5] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.443 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:30.971.451 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.460 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.468 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.526 [graph_prepare.cc:1983][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [279] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.971.548 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.971.559 [graph_prepare.cc:1984][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.971.573 [graph_prepare.cc:1985][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.971.591 [graph_prepare.cc:1986][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.971.603 [graph_prepare.cc:1987][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.971.616 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.971.628 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.971.641 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.971.717 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.729 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.738 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.747 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.755 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DropOutPass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.764 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.772 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.780 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.789 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.797 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.805 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.813 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.821 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.829 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.838 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.846 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.971.866 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.971.878 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.971.904 [graph_prepare.cc:1988][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [292] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.971.917 [graph_manager.cc:1065][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [724] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.984.386 [graph_manager.cc:1077][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12451] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.984.434 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.984.461 [graph_manager.cc:1080][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.225 [graph_manager.cc:1081][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [2739] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.267 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.283 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.294 [graph_manager.cc:1082][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.323 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.336 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.349 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.463 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [104] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.479 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.492 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.505 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.550 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.568 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.583 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.601 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.633 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.646 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.655 [graph_manager.cc:2700][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [337] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.796 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.987.809 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.987.819 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.987.828 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.987.844 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.987.853 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CastRemovePass is [43] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.987.862 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.987.870 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [14] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.987.879 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [15] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.987.887 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.987.895 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [5] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.987.903 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [0] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.987.912 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.987.920 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.987.929 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.987.939 [graph_manager.cc:2741][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [265] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.948 [graph_manager.cc:2752][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.971 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.982 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.987.998 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.012 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.022 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.034 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.068 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [24] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.083 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.098 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.109 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.136 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.149 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.160 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.172 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.181 [graph_manager.cc:2810][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [213] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.205 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.988.216 [graph_manager.cc:2821][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [26] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.244 [graph_manager.cc:1087][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [933] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.377 [graph_manager.cc:1088][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [120] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.430 [graph_manager.cc:1089][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [33] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.449 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.462 [graph_manager.cc:1097][EVENT]167129 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:30.988.483 [graph_manager.cc:3325][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.653 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.668 [engine_place.cc:144][EVENT]167129 Run:The time cost of aicpu_ascend_kernel::CheckSupported is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.746 [graph_manager.cc:3351][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [251] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.762 [graph_manager.cc:3364][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.834 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.849 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.963 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [105] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.988.989 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.989.040 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.989.072 [graph_manager.cc:3405][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [297] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.989.090 [graph_manager.cc:3412][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.990.743 [graph_manager.cc:3422][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [1637] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.990.778 [graph_manager.cc:3428][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.990.906 [graph_manager.cc:3467][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [109] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.990.925 [graph_manager.cc:3377][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [2151] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.990.941 [graph_manager.cc:1106][EVENT]167129 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [2464] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.990.953 [graph_manager.cc:1115][EVENT]167129 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:30.990.975 [graph_manager.cc:1130][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.006 [graph_manager.cc:1131][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.050 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [26] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.068 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.078 [graph_manager.cc:2837][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [57] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.126 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.991.138 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.991.147 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.991.156 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.991.164 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.991.173 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:30.991.182 [graph_manager.cc:2864][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [88] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.202 [graph_manager.cc:2872][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.219 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.232 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.247 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.260 [compile_nodes_pass.cc:88][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.270 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.296 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.332 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [26] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.374 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [30] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.387 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.399 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.411 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.420 [graph_manager.cc:2927][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [204] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.452 [graph_manager.cc:2937][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.512 [graph_manager.cc:2943][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [50] micro second. [INFO] GE(164040,python):2024-01-10-11:37:30.991.527 [graph_manager.cc:2950][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.001.252 [graph_manager.cc:2958][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [33] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.001.298 [graph_manager.cc:1132][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [10278] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.001.420 [graph_manager.cc:1135][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [106] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.001.462 [graph_manager.cc:2975][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.001.631 [graph_manager.cc:2981][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [146] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.001.649 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.001.661 [graph_manager.cc:2986][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.001.670 [graph_manager.cc:1136][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [230] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.001.799 [graph_manager.cc:3555][EVENT]167129 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [59] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.001.888 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.001.904 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.001.978 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [64] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.002.000 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.002.032 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.002.053 [graph_builder.cc:865][EVENT]167129 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [193] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.002.125 [graph_builder.cc:288][EVENT]167129 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [57] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.002.258 [graph_builder.cc:293][EVENT]167129 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [120] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.002.540 [model_builder.cc:1133][EVENT]167129 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [163] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.002.809 [block_mem_assigner.cc:4069][EVENT]167248 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164040,python):2024-01-10-11:37:31.002.809 [block_mem_assigner.cc:4069][EVENT]167249 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164040,python):2024-01-10-11:37:31.003.131 [model_builder.cc:1144][EVENT]167129 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [568] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.003.160 [model_builder.cc:1152][EVENT]167129 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.003.176 [model_builder.cc:1157][EVENT]167129 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.003.323 [model_builder.cc:1167][EVENT]167129 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [135] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.003.343 [model_builder.cc:1174][EVENT]167129 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.003.373 [model_builder.cc:1180][EVENT]167129 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.003.438 [model_builder.cc:1184][EVENT]167129 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [52] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.003.459 [graph_builder.cc:304][EVENT]167129 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [1173] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:31.003.696 [logger.cc:1071] 167129 ModelBindStream: model_id=576, stream_id=1857, flag=0. [INFO] GE(164040,python):2024-01-10-11:37:31.003.777 [task_generator.cc:804][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.003.860 [task_generator.cc:805][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [67] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.004.475 [task_generator.cc:814][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [589] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.004.491 [task_generator.cc:954][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [719] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.004.556 [task_generator.cc:967][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [40] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:31.004.575 [logger.cc:1084] 167129 ModelUnbindStream: model_id=576, stream_id=1857, [INFO] GE(164040,python):2024-01-10-11:37:31.004.635 [graph_builder.cc:310][EVENT]167129 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [1163] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.004.759 [graph_manager.cc:1152][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3037] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.004.776 [graph_manager.cc:1164][EVENT]167129 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:31.004.808 [graph_manager.cc:1271][EVENT]167129 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [44899] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.004.819 [graph_manager.cc:1272][EVENT]167129 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:37:31.005.127 [atrace_api.c:93](tid:167129) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:37:31.005.150 [atrace_api.c:95](tid:167129) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:37:31.005.843 [model_introduction.cc:236][EVENT]167129 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(164040,python):2024-01-10-11:37:31.005.867 [model_introduction.cc:294][EVENT]167129 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(164040,python):2024-01-10-11:37:31.005.879 [model_introduction.cc:366][EVENT]167129 Data:model io_info size:0 [INFO] GE(164040,python):2024-01-10-11:37:31.008.229 [graph_converter.cc:838][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [701] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.008.292 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.008.550 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [240] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.008.610 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.008.624 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [54] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.008.662 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.008.686 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.008.705 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.008.739 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [26] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.008.782 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.008.791 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [41] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.008.808 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.008.827 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.008.840 [graph_converter.cc:849][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [572] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.008.956 [graph_converter.cc:853][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [106] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.009.414 [graph_converter.cc:857][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [444] micro second. [INFO] GE(164040,python):2024-01-10-11:37:31.009.504 [graph_converter.cc:862][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [69] micro second. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:37:31.010.951 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_0_LoopCountActor) running, loop count: 1, current count: 1, total running count: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:31.011.357 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_0 execution count: 1, execution time: 136.968 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:37:31.011.447 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.011.592 [mindspore/ccsrc/runtime/device/kernel_runtime_manager.cc:35] ClearGraphResource] Clear device Ascend_1 graph 0 runtime resource [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:31.014.845 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:198] Compile] Input plan: +-Transfer,send_epoch_end:false,total_batch:2340) | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:31.014.966 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:216] Compile] Plan before optimization: +-Top | +-Transfer,send_epoch_end:false,total_batch:2340) | | +-Repeat(count:1) | | | +-Batch(batch_size:32 drop_remainder:true) | | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:31.014.999 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:60] PrePass] Running pre pass loops. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:31.015.019 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:31.015.053 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:31.015.135 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:31.015.163 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:31.015.177 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:31.015.202 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:31.015.215 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/cache_transform_pass.cc:182] RunOnTree] Pre pass: Cache transform pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:31.015.238 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/cache_transform_pass.cc:199] RunOnTree] Pre pass: Cache transform pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:31.015.250 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:91] PrePass] Pre pass offload complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:31.015.264 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:116] PostPass] Running post pass loops. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:31.015.296 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:135] PostPass] Post passes complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:31.015.332 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:230] Compile] Plan after optimization: +-Top | +-Transfer,send_epoch_end:false,total_batch:2340) | | +-EpochCtrl(epoch:5) | | | +-Batch(batch_size:32 drop_remainder:true) | | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | | +-MnistDataset [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.016.026 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_data_queue.cc:227] AscendTdtQueue] Select MBUF channel, the capacity of data queue is: 128 [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:37:31.016.093 [mindspore/ccsrc/minddata/dataset/engine/datasetops/epoch_ctrl_op.cc:25] EpochCtrlOp] Welcome to Epoch Ctrl Op. [INFO] MD(164040,fffdda7fc0f0,python):2024-01-10-11:37:31.019.056 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at train-labels-idx1-ubyte. [INFO] MD(164040,fffdda7fc0f0,python):2024-01-10-11:37:31.019.111 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at train-images-idx3-ubyte. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.023.371 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:456] SendDataToAscend] Device queue, sending data to Ascend. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.377.614 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:493] GenerateArgumentsKey] Generate a new compile key for new args, key: 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.377.776 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:501] GenerateArgumentsKey] New cached args: [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.378.658 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:978] CompileInner] Start compiling, phase: train.1704857851222130944.281471961594384.0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.378.693 [mindspore/ccsrc/pipeline/jit/ps/event_message_print.cc:37] PrintEventMessage] Start compiling '_DataWrapper.construct' and it will take a while. Please wait... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.388.220 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1659] VmPipeline] This worker is initialized. No need to add worker action. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:37:31.388.309 [mindspore/ccsrc/backend/graph_compiler/transform.cc:573] CreateBackend] CreateBackend is: ge [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.388.337 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.388.352 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEBUG(164040,ffffb1c2c440,python):2024-01-10-11:37:31.388.529 [mindspore/ccsrc/debug/debugger/debugger.cc:129] Init] Debugger got device_id: 1 [INFO] DEBUG(164040,ffffb1c2c440,python):2024-01-10-11:37:31.388.550 [mindspore/ccsrc/debug/debugger/debugger.cc:131] Init] Debugger got device_target: Ascend [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.388.573 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1311] Run] Pipeline run [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.388.596 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start parse action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.399.887 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end parse action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.399.950 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start symbol_resolve action. [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.419.382 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] _no_sens_impl_4 update var `grads` with node @_no_sens_impl_4:grads{[0]: CNode_5, [1]: grads} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.419.628 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_no_sens_impl_4] Added global python symbol: {F : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.420.110 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] _no_sens_impl_4 update var `loss` with node @_no_sens_impl_4:loss{[0]: CNode_6, [1]: loss, [2]: CNode_7} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.437.591 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] mindspore_nn_optim_momentum_Momentum_construct_8 update var `gradients` with node @mindspore_nn_optim_momentum_Momentum_construct_8:gradients{[0]: CNode_9, [1]: param_gradients} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.437.987 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] mindspore_nn_optim_momentum_Momentum_construct_8 update var `gradients` with node @mindspore_nn_optim_momentum_Momentum_construct_8:gradients{[0]: CNode_10, [1]: gradients} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.438.271 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] mindspore_nn_optim_momentum_Momentum_construct_8 update var `gradients` with node @mindspore_nn_optim_momentum_Momentum_construct_8:gradients{[0]: CNode_11, [1]: gradients} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.438.545 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] mindspore_nn_optim_momentum_Momentum_construct_8 update var `gradients` with node @mindspore_nn_optim_momentum_Momentum_construct_8:gradients{[0]: CNode_12, [1]: gradients} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.439.279 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_13{[0]: CNode_14, [1]: CNode_15, [2]: CNode_16}, block: 0x2add8170/mindspore_nn_optim_momentum_Momentum_construct_8, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/nn/optim/momentum.py:211/ self.assignadd(self.global_step, self.global_step_increase_tensor)/ [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.439.878 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_optim_momentum_Momentum_construct_8] Added global python symbol: {F : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.440.128 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_optim_momentum_Momentum_construct_8] Added global python symbol: {_momentum_opt : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.440.835 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_17{[0]: ValueNode Depend, [1]: CNode_18, [2]: CNode_19}, state: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_13{[0]: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_14{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.optim.momentum..', [2]: ValueNode assignadd}, [1]: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_15{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.optim.momentum..', [2]: ValueNode global_step}, [2]: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_16{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.optim.momentum..', [2]: ValueNode global_step_increase_tensor}} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.455.388 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20] Added global python symbol: {F : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.455.644 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20] Added global python symbol: {_get_datatype : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.456.162 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20] Added global python symbol: {_cast_datatype : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.456.348 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20] Added global python symbol: {mstype : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.456.634 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20 update var `grads` with node @mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20:grads{[0]: CNode_21, [1]: CNode_22, [2]: param_grads} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.457.304 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20] Added global python symbol: {reduce_opt : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.467.467 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23] Added global python symbol: {_check_is_tensor : Prim[_check_is_tensor]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.467.986 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_24{[0]: CNode_25, [1]: ValueNode logits, [2]: param_logits, [3]: CNode_26}, block: 0x2add8170/mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/nn/loss/loss.py:777/ _check_is_tensor('logits', logits, self.cls_name)/ [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.468.524 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_27{[0]: CNode_25, [1]: ValueNode labels, [2]: param_labels, [3]: CNode_28}, block: 0x2add8170/mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/nn/loss/loss.py:778/ _check_is_tensor('labels', labels, self.cls_name)/ [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.469.153 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_29{[0]: ValueNode Depend, [1]: CNode_30, [2]: CNode_31}, state: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_32{[0]: ValueNode MakeTuple, [1]: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_24{[0]: CNode_25, [1]: ValueNode logits, [2]: param_logits, [3]: CNode_26}, [2]: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_27{[0]: CNode_25, [1]: ValueNode labels, [2]: param_labels, [3]: CNode_28}} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.478.255 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_34, [1]: param_x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.478.550 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_35, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.478.838 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_36, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.479.110 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_37, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.479.377 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_38, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.479.636 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_39, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.479.904 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_40, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.480.171 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_41, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.480.445 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_42, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.480.721 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_43, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.481.038 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_44, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.481.311 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_45, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.492.869 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_46] Added global python symbol: {len : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.493.037 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.493.381 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.493.562 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.494.055 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_46:CNode_48{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.494.198 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_47:x{[0]: CNode_49, [1]: param_фx, [2]: CNode_48} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.494.668 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_46:CNode_50{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.495.231 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_47] Added global python symbol: {len : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.495.297 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_46:CNode_51{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.495.406 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_46:x_shape{[0]: CNode_52, [1]: param_x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.495.545 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.495.835 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.496.117 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_47] Added global python symbol: {F : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.496.208 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_46] Added global python symbol: {F : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.496.260 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_46:CNode_53{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.496.579 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.503.203 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_54] Added global python symbol: {len : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.503.370 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.503.700 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.503.873 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.504.359 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_54:CNode_56{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.504.515 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_55:x{[0]: CNode_57, [1]: param_фx, [2]: CNode_56} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.504.952 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_54:CNode_58{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.505.404 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_55] Added global python symbol: {len : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.505.469 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_54:CNode_59{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.505.575 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_54:x_shape{[0]: CNode_60, [1]: param_x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.505.732 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.506.038 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.506.317 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_55] Added global python symbol: {F : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.506.407 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_54] Added global python symbol: {F : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.506.461 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_54:CNode_61{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.506.778 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.510.581 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_62] Added global python symbol: {len : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.510.743 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.511.074 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.511.241 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.511.714 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_62:CNode_64{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.511.853 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_63:x{[0]: CNode_65, [1]: param_фx, [2]: CNode_64} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.512.289 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_62:CNode_66{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.512.758 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_63] Added global python symbol: {len : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.512.824 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_62:CNode_67{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.512.934 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_62:x_shape{[0]: CNode_68, [1]: param_x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.513.070 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.513.360 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.513.650 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_63] Added global python symbol: {F : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.513.790 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_62] Added global python symbol: {F : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.513.845 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_62:CNode_69{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.514.165 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.517.995 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Flatten_construct_70] Added global python symbol: {F : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.531.630 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1261] ParseNameConstant] The NameConstant is bool:False [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.531.936 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:3 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.532.194 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.532.323 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1261] ParseNameConstant] The NameConstant is bool:True [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.533.037 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_pooling_MaxPool2d_construct_71] Added global python symbol: {isinstance : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.533.141 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_pooling_MaxPool2d_construct_72] Added global python symbol: {isinstance : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.533.201 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_pooling_MaxPool2d_construct_71 update var `isinstance` with node @mindspore_nn_layer_pooling_MaxPool2d_construct_72:CNode_73{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.pooling', [2]: ValueNode isinstance} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.533.363 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_pooling_MaxPool2d_construct_71] Added global python symbol: {tuple : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.533.445 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_pooling_MaxPool2d_construct_72] Added global python symbol: {tuple : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.533.494 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_pooling_MaxPool2d_construct_71 update var `tuple` with node @mindspore_nn_layer_pooling_MaxPool2d_construct_72:CNode_74{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.pooling', [2]: ValueNode tuple} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.533.869 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.533.992 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.534.198 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.534.311 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.534.592 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.584.167 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.584.349 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.585.047 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @canonicalize_axis_75:CNode_76{[0]: ValueNode check_axis_valid_77, [1]: param_axis, [2]: ndim}, block: 0x33fec130/canonicalize_axis_75, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1606/ check_axis_valid(axis, ndim)/ [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.585.234 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.585.526 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @canonicalize_axis_75:CNode_78{[0]: ValueNode Depend, [1]: CNode_79, [2]: CNode_80}, state: @canonicalize_axis_75:CNode_76{[0]: ValueNode check_axis_valid_77, [1]: @canonicalize_axis_75:param_axis, [2]: @canonicalize_axis_75:ndim{[0]: CNode_81}} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.585.859 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {isinstance : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.586.020 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {Tensor : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.586.591 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {int : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.587.053 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {bool : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.587.793 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {check_flatten_order_const : Prim[check_flatten_order]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.588.396 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @2↓flatten_83:CNode_84{[0]: CNode_85, [1]: param_order}, block: 0x340138b0/2↓flatten_83, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1615/ check_flatten_order_const(order)/ [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.588.822 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {rank_ : Prim[Rank]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.589.174 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.589.234 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.589.466 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {reshape_ : Prim[Reshape]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.589.650 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.589.988 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {ops : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.590.211 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.590.734 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {transpose_ : Prim[Transpose]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.591.169 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_86] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.591.277 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.591.345 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `shape_` with node @flatten_82:CNode_87{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode shape_} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.591.681 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_86] Added global python symbol: {rank_ : Prim[Rank]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.591.755 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `rank_` with node @flatten_82:CNode_88{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode rank_} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.592.051 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `start_dim` with node @flatten_82:param_start_dim [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.592.206 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.592.354 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `end_dim` with node @flatten_82:param_end_dim [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.592.474 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.592.759 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.592.962 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.593.274 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_86] Added global python symbol: {reshape_ : Prim[Reshape]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.593.352 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `reshape_` with node @flatten_82:CNode_89{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode reshape_} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.593.570 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.593.923 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_86] Added global python symbol: {flatten_ : Prim[Flatten]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.594.037 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {flatten_ : Prim[Flatten]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.594.110 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `flatten_` with node @flatten_82:CNode_90{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode flatten_} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.594.462 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `canonicalize_axis` with node ValueNode canonicalize_axis_75 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.594.950 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `check_dim_valid` with node ValueNode check_dim_valid_91 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.595.497 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @4↓flatten_92:CNode_93{[0]: ValueNode check_dim_valid_91, [1]: start_dim, [2]: end_dim}, block: 0x3403fb70/4↓flatten_92, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1636/ check_dim_valid(start_dim, end_dim)/ [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.595.731 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.595.787 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.596.053 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.596.505 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.597.032 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.597.618 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.598.168 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @2↓flatten_83:CNode_94{[0]: ValueNode Depend, [1]: CNode_95, [2]: CNode_96}, state: @2↓flatten_83:CNode_84{[0]: @flatten_82:CNode_85{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode check_flatten_order_const}, [1]: @flatten_82:param_order} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.598.290 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @4↓flatten_92:CNode_97{[0]: ValueNode Depend, [1]: CNode_98, [2]: CNode_99}, state: @4↓flatten_92:CNode_93{[0]: ValueNode check_dim_valid_91, [1]: @4↓flatten_92:idx{[0]: ValueNode canonicalize_axis_75, [1]: param_start_dim, [2]: x_rank}, [2]: @4↓flatten_92:end_dim{[0]: ValueNode canonicalize_axis_75, [1]: param_end_dim, [2]: x_rank}} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.598.393 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.598.485 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.599.644 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:174] CheckFuncReturn] Function must has 'return' statement, but missing in function 'flatten' at /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1557. FuncGraph: ↓check_dim_valid_100. We will add a 'return None' statement automatically. [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.599.754 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:174] CheckFuncReturn] Function must has 'return' statement, but missing in function 'flatten' at /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1557. FuncGraph: ↓check_axis_valid_101. We will add a 'return None' statement automatically. [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.644.125 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [shape_102] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.652.939 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end symbol_resolve action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.652.998 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start graph_reusing action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.653.017 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.basic.DenseDense[True, None]_ID [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.653.030 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.conv.Conv2dConv2dodict_values([6, 16, 5, 1, 'valid', 0, False, ])_ID [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.653.041 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.conv.Conv2dConv2dodict_values([1, 6, 5, 1, 'valid', 0, False, ])_ID [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.653.056 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end graph_reusing action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.653.075 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start meta_unpack_prepare action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.654.280 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end meta_unpack_prepare action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.654.326 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pre_cconv action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.654.344 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pre_cconv action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.654.362 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start abstract_specialize action. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.673.921 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:502] SendDataToAscend] Begin to send data to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.673.990 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:1182] PrintBeginInfoWhenFirstBatch] Loading dataset and begin to push first batch into device ... [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.675.609 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:1192] PrintEndInfoWhenFirstBatch] Loading dataset and push first batch into device successful. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.675.640 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.676.182 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.676.663 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 3 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.677.367 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_106{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.677.457 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.677.469 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 4 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.678.522 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 5 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.678.590 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_108{[0]: CNode_109}, new_node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_110{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.678.650 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_108{[0]: CNode_109}, new node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_108{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.679.520 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 6 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.680.518 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 7 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.681.499 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 8 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.681.512 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_equal_scalar_112] Added global python symbol: {F : } [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.681.964 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 4, Shape: NoShape), AbstractScalar(Type: Int64, Value: 3, Shape: NoShape)}, g: _equal_scalar_112 [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.682.522 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 9 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.682.766 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_113:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_113:CNode_115{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.682.835 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_113:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_113:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.683.613 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 10 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.684.714 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 11 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.685.614 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 12 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.686.360 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_117{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.686.428 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.686.681 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 13 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.686.766 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_118{[0]: CNode_119}, new_node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_120{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.686.821 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_118{[0]: CNode_119}, new node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_118{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.687.484 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_121:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_121:CNode_122{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.687.555 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_121:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_121:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.687.769 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 14 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.688.820 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 15 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.689.886 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 16 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.690.837 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 17 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.691.868 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 18 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.692.908 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 19 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.694.102 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 20 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.694.773 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_logical_not_scala_124] Added global python symbol: {auto_generate : } [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.695.049 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 21 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.695.243 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Bool, Value: true, Shape: NoShape)}, g: _logical_not_scala_124 [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.696.084 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 22 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.697.149 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 23 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.698.244 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 24 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.699.303 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 25 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.700.397 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 26 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.700.724 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bool_not_125] Added global python symbol: {_get_cache_prim : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.700.877 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bool_not_125] Added global python symbol: {BoolNot : } [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.701.487 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 27 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.702.460 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 28 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.703.470 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 29 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.704.464 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 30 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.705.500 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 31 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.706.650 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 32 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.707.542 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 33 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.708.537 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 34 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.709.803 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 35 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.710.611 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 36 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.711.651 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 37 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.712.697 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 38 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.713.778 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 39 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.714.822 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 40 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.715.018 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_126] Added global python symbol: {str : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.715.474 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↻_get_cache_prim_for_pynative_127] Added global python symbol: {str : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.715.758 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↻_get_cache_prim_for_pynative_127 update var `str` with node @↵_get_cache_prim_for_pynative_128:param_фstr [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.715.809 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 41 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.716.005 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_126] Added global python symbol: {tuple : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.716.213 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] _get_cache_prim_for_pynative_129 update var `key` with node @_get_cache_prim_for_pynative_129:key{[0]: CNode_130, [1]: key, [2]: CNode_131} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.716.853 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 42 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.716.993 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_132] Added global python symbol: {str : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.717.614 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_132] Added global python symbol: {_PRIM_CACHE : {}} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.717.729 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_126] Added global python symbol: {_PRIM_CACHE : {}} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.717.853 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 43 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.718.004 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_132] Added global python symbol: {Primitive : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.718.101 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_126] Added global python symbol: {Primitive : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.718.825 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @✓↓_get_cache_prim_for_pynative_133:CNode_134{[0]: ValueNode MetaFuncGraph-unpack_call.135, [1]: CNode_136, [2]: param_фargs, [3]: param_фkwargs}, block: 0x315ea140/✓↓_get_cache_prim_for_pynative_133, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/_primitive_cache.py:84/ prim.__init__(*args, **kwargs)/ [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.718.961 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 44 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.719.445 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 2↓_get_cache_prim_for_pynative_137 update var `key` with node @↓_get_cache_prim_for_pynative_138:key{[0]: param_фstr, [1]: param_фkey} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.719.610 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @↻_get_cache_prim_for_pynative_139:CNode_140{[0]: ValueNode Depend, [1]: CNode_141, [2]: CNode_142}, state: @↻_get_cache_prim_for_pynative_139:CNode_143{[0]: ValueNode MetaFuncGraph-add.144, [1]: @↵_get_cache_prim_for_pynative_132:param_@CNode_143, [2]: ValueNode 1} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.719.725 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @✓↓_get_cache_prim_for_pynative_133:CNode_145{[0]: ValueNode Depend, [1]: CNode_146, [2]: CNode_147}, state: @✓↓_get_cache_prim_for_pynative_133:CNode_134{[0]: ValueNode MetaFuncGraph-unpack_call.135, [1]: @✓↓_get_cache_prim_for_pynative_133:CNode_136{[0]: ValueNode getattr, [1]: prim, [2]: ValueNode __init__}, [2]: @↵_get_cache_prim_for_pynative_132:param_фargs, [3]: @↵_get_cache_prim_for_pynative_132:param_фkwargs} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.719.959 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 45 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.720.967 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 46 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.721.121 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_148:CNode_149{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.721.189 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_148:CNode_150{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.721.233 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_148:CNode_151{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.722.061 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BoolNot. node: @bool_not_125:CNode_152{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x}, new_node: @bool_not_125:CNode_153{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.722.064 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 47 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.722.126 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BoolNot. node: @bool_not_125:CNode_152{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x}, new node: @bool_not_125:CNode_152{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.723.145 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 48 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.724.103 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 49 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.725.104 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 50 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.726.248 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 51 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.727.215 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 52 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.728.244 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 53 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.728.281 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_equal_string_154] Added global python symbol: {F : } [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.728.668 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: String, Value: C, Shape: NoShape), AbstractScalar(Type: String, Value: F, Shape: NoShape)}, g: _equal_string_154 [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.729.159 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 54 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.730.168 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 55 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.730.269 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_155:CNode_156{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_155:CNode_157{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.730.343 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_155:CNode_156{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_155:CNode_156{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.731.188 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 56 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.732.162 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 57 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.732.989 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_neg_scalar_159] Added global python symbol: {F : } [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.733.192 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 58 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.733.340 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 1, Shape: NoShape)}, g: _neg_scalar_159 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.734.080 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarUsub. node: @_neg_scalar_160:CNode_161{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x}, new_node: @_neg_scalar_160:CNode_162{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.734.157 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarUsub. node: @_neg_scalar_160:CNode_161{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x}, new node: @_neg_scalar_160:CNode_161{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.734.395 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 59 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.734.870 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_163:CNode_164{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_163:CNode_165{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.734.939 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_163:CNode_164{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_163:CNode_164{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.735.266 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 60 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.735.403 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @3↓flatten_166:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput}, new_node: @3↓flatten_166:CNode_167{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.735.470 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @3↓flatten_166:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput}, new node: @3↓flatten_166:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.736.328 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 61 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.737.337 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 62 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.738.362 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 63 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.739.382 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 64 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.739.624 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_169] Added global python symbol: {F : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.740.354 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_169] Added global python symbol: {InSequence : } [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.740.469 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 65 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.740.756 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_169] Added global python symbol: {const_utils : } [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.741.285 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 4, Shape: NoShape), AbstractTuple{ element[0]: AbstractScalar(Type: Int64, Value: 0, Shape: NoShape), element[1]: AbstractScalar(Type: Int64, Value: 1, Shape: NoShape), sequence_nodes: {@✓3↓flatten_170:CNode_171{[0]: ValueNode MakeTuple, [1]: ValueNode 0, [2]: ValueNode 1}, elements_use_flags: {ptr: 0x31650ab0, value: [const vector]{0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: _number_in_tuple_169 [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.741.369 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 66 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.742.473 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 67 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.743.420 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 68 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.744.416 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 69 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.744.958 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Flatten. node: @↓✓3↓flatten_172:CNode_173{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput}, new_node: @↓✓3↓flatten_172:CNode_174{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.745.030 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Flatten. node: @↓✓3↓flatten_172:CNode_173{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput}, new node: @↓✓3↓flatten_172:CNode_173{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.745.390 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_175:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_175:CNode_176{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.745.444 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_175:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_175:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.745.548 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 70 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.746.642 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 71 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.747.603 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 72 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.748.425 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_not_equal_scalar_178] Added global python symbol: {F : } [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.748.678 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 73 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.748.813 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 2, Shape: NoShape), AbstractScalar(Type: Int64, Value: 2, Shape: NoShape)}, g: _not_equal_scalar_178 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.749.654 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_179:CNode_180{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_179:CNode_181{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.749.666 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 74 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.749.786 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_179:CNode_180{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_179:CNode_180{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.750.744 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 75 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.751.721 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 76 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.751.897 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_183:CNode_184{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_183:CNode_185{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.751.968 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_183:CNode_184{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_183:CNode_184{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.752.827 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 77 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.753.231 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_186:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_186:CNode_187{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias, [3]: ValueNode 0} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.753.306 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_186:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_186:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias, [3]: ValueNode 0} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.753.664 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_188{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.753.768 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.753.810 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 78 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.753.969 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_189:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_189:CNode_190{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.754.020 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_189:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_189:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.754.876 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 79 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.754.915 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_191:CNode_192{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_191:CNode_193{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.754.984 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_191:CNode_192{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_191:CNode_192{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.755.898 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 80 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.756.940 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 81 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.757.048 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_194:CNode_195{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_194:CNode_196{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.757.119 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_194:CNode_195{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_194:CNode_195{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.758.085 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 82 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.758.462 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_197:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_197:CNode_198{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias, [3]: ValueNode 0} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.758.534 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_197:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_197:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias, [3]: ValueNode 0} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.758.812 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_199{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.758.866 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.759.050 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 83 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.759.054 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_200:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_200:CNode_201{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.759.144 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_200:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_200:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.759.910 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 84 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.760.037 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_202:CNode_203{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_202:CNode_204{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.760.106 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_202:CNode_203{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_202:CNode_203{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.760.862 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 85 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.762.073 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_205:CNode_206{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_205:CNode_207{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.762.088 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 86 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.762.153 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_205:CNode_206{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_205:CNode_206{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.763.230 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 87 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.763.422 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_208:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_208:CNode_209{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias, [3]: ValueNode 0} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.763.495 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_208:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_208:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias, [3]: ValueNode 0} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.764.311 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 88 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.765.437 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 89 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.766.583 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 90 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.766.623 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny)}, g: hyper_map_212 [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.767.788 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 91 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.768.941 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 92 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.769.141 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_ones_like_tensor_213] Added global python symbol: {P : } [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.769.526 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny)}, g: _ones_like_tensor_213 [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.770.168 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 93 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.770.740 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: OnesLike. node: @_ones_like_tensor_214:grads{[0]: ValueNode PrimFunc_OnesLike, [1]: param_x}, new_node: @_ones_like_tensor_214:CNode_215{[0]: ValueNode PrimFunc_OnesLike, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.770.807 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_OnesLike. node: @_ones_like_tensor_214:grads{[0]: ValueNode PrimFunc_OnesLike, [1]: param_x}, new node: @_ones_like_tensor_214:grads{[0]: ValueNode PrimFunc_OnesLike, [1]: param_x} [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.771.217 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: env_get(AbstractScalar(Type: Object:EnvType, Value: ValueAny, Shape: NoShape), )}, AbstractTuple{ element[0]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[1]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[2]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[3]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[4]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[5]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[6]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[7]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), sequence_nodes: {@_no_sens_impl_216:CNode_217{[0]: ValueNode MakeTuple, [1]: param_conv1.weight, [2]: param_conv2.weight, [3]: param_fc1.weight, [4]: param_fc1.bias, [5]: param_fc2.weight, [6]: param_fc2.bias, [7]: param_fc3.weight, [8]: param_fc3.bias}, elements_use_flags: {ptr: 0x31738100, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: hyper_map_218 [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.771.341 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 94 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.771.616 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: env_get(AbstractScalar(Type: Object:EnvType, Value: ValueAny, Shape: NoShape), )}, AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny)}, g: hyper_map_219 [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.772.512 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 95 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.773.539 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 96 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.774.596 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensor_env_get_221] Added global python symbol: {environ_get : Prim[EnvironGet]} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.774.735 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 97 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.774.829 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensor_env_get_221] Added global python symbol: {ref_to_embed : Prim[RefToEmbed]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.775.104 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensor_env_get_221] Added global python symbol: {tensor_zeros_like : Prim[ZerosLike]} [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.775.447 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Object:EnvType, Value: ValueAny, Shape: NoShape), AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny)}, g: _tensor_env_get_221 [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.775.994 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 98 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.776.512 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_222:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_222:CNode_223{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.776.581 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_222:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_222:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.777.111 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 99 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.777.778 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_224:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_224:CNode_225{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.777.853 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_224:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_224:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:31.778.460 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 100 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.778.971 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_226:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_226:CNode_227{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.779.041 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_226:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_226:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.780.156 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_228:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_228:CNode_229{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.780.240 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_228:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_228:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.781.408 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_230:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_230:CNode_231{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.781.477 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_230:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_230:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.782.588 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_232:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_232:CNode_233{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.782.658 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_232:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_232:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.783.767 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_234:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_234:CNode_235{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.783.835 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_234:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_234:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.784.934 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_236:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_236:CNode_237{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.785.003 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_236:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_236:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.785.942 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: S-signature()}, AbstractTuple{ element[0]: AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[1]: AbstractTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[2]: AbstractTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[3]: AbstractTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[4]: AbstractTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[5]: AbstractTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[6]: AbstractTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[7]: AbstractTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), sequence_nodes: {@hyper_map_239:grads{[0]: ValueNode MakeTuple, [1]: grads, [2]: grads, [3]: grads, [4]: grads, [5]: grads, [6]: grads, [7]: grads, [8]: grads}, elements_use_flags: {ptr: 0x317db650, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: map_240 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.788.900 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensors_get_datatype_242] Added global python symbol: {F : } [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.789.263 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny)}, g: _tensors_get_datatype_242 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.798.638 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensors_cast_datatype_244] Added global python symbol: {F : } [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.799.022 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractType(Value: Float32), AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny)}, g: _tensors_cast_datatype_244 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.807.106 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: S-signature(AbstractTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2df2f720, value: Tensor(shape=[], dtype=Float32, value=0.25)), AbstractScalar(Type: Bool, Value: true, Shape: NoShape), Prim: S_Prim_AllGather, Prim: S_Prim_AllReduce, )}, AbstractTuple{ element[0]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[1]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[2]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[3]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[4]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[5]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[6]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[7]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), sequence_nodes: {ValueNode (true, true, true, true, true, true, true, true), elements_use_flags: {ptr: 0x318c8af0, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} , AbstractTuple{ element[0]: AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[1]: AbstractTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[2]: AbstractTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[3]: AbstractTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[4]: AbstractTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[5]: AbstractTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[6]: AbstractTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[7]: AbstractTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), sequence_nodes: {@map_245:grads{[0]: ValueNode MakeTuple, [1]: grads, [2]: grads, [3]: grads, [4]: grads, [5]: grads, [6]: grads, [7]: grads, [8]: grads}, elements_use_flags: {ptr: 0x318a7470, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: map_246 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.814.512 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensors_allreduce_248] Added global python symbol: {F : } [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.815.640 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2df2f720, value: Tensor(shape=[], dtype=Float32, value=0.25)), AbstractScalar(Type: Bool, Value: true, Shape: NoShape), Prim: S_Prim_AllGather, Prim: S_Prim_AllReduce, AbstractScalar(Type: Bool, Value: true, Shape: NoShape), AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny)}, g: _tensors_allreduce_248 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.818.666 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_249:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_249:CNode_250{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.818.750 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_249:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_249:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.821.174 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_251:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_251:CNode_252{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.821.258 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_251:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_251:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.823.629 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_253:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_253:CNode_254{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.823.714 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_253:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_253:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.826.115 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_255:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_255:CNode_256{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.826.198 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_255:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_255:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.828.526 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_257:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_257:CNode_258{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.828.610 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_257:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_257:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.830.956 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_259:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_259:CNode_260{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.831.039 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_259:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_259:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.833.344 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_261:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_261:CNode_262{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.833.445 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_261:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_261:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.835.795 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_263:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_263:CNode_264{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.835.878 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_263:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_263:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.836.587 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: S-signature()}, AbstractTuple{ element[0]: AbstractType(Value: Float32), element[1]: AbstractType(Value: Float32), element[2]: AbstractType(Value: Float32), element[3]: AbstractType(Value: Float32), element[4]: AbstractType(Value: Float32), element[5]: AbstractType(Value: Float32), element[6]: AbstractType(Value: Float32), element[7]: AbstractType(Value: Float32), sequence_nodes: {@map_265:datatypes{[0]: ValueNode MakeTuple, [1]: datatypes, [2]: datatypes, [3]: datatypes, [4]: datatypes, [5]: datatypes, [6]: datatypes, [7]: datatypes, [8]: datatypes}, elements_use_flags: {ptr: 0x31863b20, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} , AbstractTuple{ element[0]: AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[1]: AbstractTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[2]: AbstractTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[3]: AbstractTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[4]: AbstractTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[5]: AbstractTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[6]: AbstractTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[7]: AbstractTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), sequence_nodes: {@map_266:new_grad{[0]: ValueNode MakeTuple, [1]: new_grad, [2]: new_grad, [3]: new_grad, [4]: new_grad, [5]: new_grad, [6]: new_grad, [7]: new_grad, [8]: new_grad}, elements_use_flags: {ptr: 0x31a31b40, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: map_267 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.846.349 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: S-signature(Prim: S_Prim_ApplyMomentum, AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), )}, AbstractTuple{ element[0]: AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[1]: AbstractTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[2]: AbstractTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[3]: AbstractTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[4]: AbstractTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[5]: AbstractTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[6]: AbstractTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), element[7]: AbstractTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), sequence_nodes: {@map_270:new_grad{[0]: ValueNode MakeTuple, [1]: new_grad, [2]: new_grad, [3]: new_grad, [4]: new_grad, [5]: new_grad, [6]: new_grad, [7]: new_grad, [8]: new_grad}, elements_use_flags: {ptr: 0x31ab3f70, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} , AbstractTuple{ element[0]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[1]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[2]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[3]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[4]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[5]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[6]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[7]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), sequence_nodes: {@mindspore_nn_optim_momentum_Momentum_construct_271:CNode_272{[0]: ValueNode MakeTuple, [1]: param_conv1.weight, [2]: param_conv2.weight, [3]: param_fc1.weight, [4]: param_fc1.bias, [5]: param_fc2.weight, [6]: param_fc2.bias, [7]: param_fc3.weight, [8]: param_fc3.bias}, elements_use_flags: {ptr: 0x31ac98c0, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} , AbstractTuple{ element[0]: AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[1]: AbstractRefTensor(key: moments.conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[2]: AbstractRefTensor(key: moments.fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[3]: AbstractRefTensor(key: moments.fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[4]: AbstractRefTensor(key: moments.fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[5]: AbstractRefTensor(key: moments.fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[6]: AbstractRefTensor(key: moments.fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), element[7]: AbstractRefTensor(key: moments.fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), sequence_nodes: {@mindspore_nn_optim_momentum_Momentum_construct_271:CNode_273{[0]: ValueNode MakeTuple, [1]: param_moments.conv1.weight, [2]: param_moments.conv2.weight, [3]: param_moments.fc1.weight, [4]: param_moments.fc1.bias, [5]: param_moments.fc2.weight, [6]: param_moments.fc2.bias, [7]: param_moments.fc3.weight, [8]: param_moments.fc3.bias}, elements_use_flags: {ptr: 0x31aca160, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: hyper_map_274 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.846.999 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: S-signature(Prim: S_Prim_ApplyMomentum, AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), )}, AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny)}, g: hyper_map_275 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.851.234 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensor_run_opt_ext_277] Added global python symbol: {F : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.851.477 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1261] ParseNameConstant] The NameConstant is bool:True [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.851.973 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{Prim: S_Prim_ApplyMomentum, AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny)}, g: _tensor_run_opt_ext_277 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.860.678 [mindspore/ccsrc/pipeline/jit/ps/action.cc:282] AbstractAnalyze] function call depth: 0, simulate call depth: 0 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.860.901 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:222] Run] Specialize set top func graph context: {FuncGraph: mindspore_train_dataset_helper__DataWrapper_construct_103 Args: [0]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [1]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [2]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [3]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [4]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [5]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [6]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [7]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [8]: AbstractRefTensor(key: global_step ref_value: AbstractRefTensor(shape: (1), element: AbstractScalar(Type: Int32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [9]: AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [10]: AbstractRefTensor(key: moments.conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [11]: AbstractRefTensor(key: moments.fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [12]: AbstractRefTensor(key: moments.fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [13]: AbstractRefTensor(key: moments.fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [14]: AbstractRefTensor(key: moments.fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [15]: AbstractRefTensor(key: moments.fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [16]: AbstractRefTensor(key: moments.fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [17]: AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [18]: AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), Parent: } [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.861.100 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @mindspore_train_dataset_helper__DataWrapper_construct_103:CNode_278{[0]: ValueNode MetaFuncGraph-unpack_call.279, [1]: ValueNode mindspore_nn_wrap_cell_wrapper_TrainOneStepCell_construct_280, [2]: outputs}, flag: 1 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.861.445 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @UnpackCall_281:CNode_278{[0]: param_282, [1]: CNode_278, [2]: CNode_278}, flag: 1 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.861.838 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @mindspore_nn_wrap_cell_wrapper_TrainOneStepCell_construct_283:CNode_284{[0]: ValueNode ✓mindspore_nn_wrap_cell_wrapper_TrainOneStepCell_construct_285}, flag: 1 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.862.083 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @✓mindspore_nn_wrap_cell_wrapper_TrainOneStepCell_construct_285:CNode_286{[0]: ValueNode MetaFuncGraph-unpack_call.287, [1]: ValueNode _no_sens_impl_288, [2]: CNode_289}, flag: 1 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.862.347 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @UnpackCall_290:CNode_286{[0]: param_291, [1]: CNode_286, [2]: CNode_286}, flag: 1 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.862.539 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @_no_sens_impl_216:CNode_292{[0]: ValueNode ✗_no_sens_impl_293}, flag: 1 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.862.639 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @✗_no_sens_impl_293:CNode_294{[0]: ValueNode ↓_no_sens_impl_295}, flag: 1 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.862.726 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @_no_sens_impl_216:loss{[0]: ValueNode S_Prim_Depend, [1]: loss, [2]: CNode_7}, flag: 1 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.862.784 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @_no_sens_impl_216:CNode_7{[0]: ValueNode mindspore_nn_optim_momentum_Momentum_construct_271, [1]: grads}, flag: 1 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.868.856 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @mindspore_nn_optim_momentum_Momentum_construct_271:CNode_17{[0]: ValueNode Depend, [1]: CNode_18, [2]: CNode_19}, flag: 1 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.896.879 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end abstract_specialize action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.896.952 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pack_expand action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.897.391 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pack_expand action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.897.434 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start auto_monad action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.900.844 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end auto_monad action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.900.897 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start inline action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.900.917 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end inline action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.900.937 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pre_auto_parallel action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.900.979 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pre_auto_parallel action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.901.000 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pipeline_split action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.901.014 [mindspore/ccsrc/pipeline/jit/ps/pipeline_split.cc:247] PipelineSplit] Only auto_parallel and semi_auto_parallel support pipeline split. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.901.026 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pipeline_split action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.901.043 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start optimize action. [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.912.363 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [fill_296] Added global python symbol: {cast_ : Prim[Cast]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.912.624 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] fill_296 update var `value` with node @fill_296:value{[0]: CNode_297, [1]: param_value, [2]: param_type} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.912.884 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [fill_296] Added global python symbol: {fillv2_ : Prim[FillV2]} [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:37:31.937.488 [mindspore/ccsrc/frontend/parallel/pynative_shard/pynative_shard.cc:334] PynativeShard] Only auto_parallel and semi_auto_parallel support pynative shard [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:37:31.937.565 [mindspore/ccsrc/frontend/parallel/step_parallel.cc:3009] StepParallel] Strategies would be ignored in data_parallel, shard() only valid in [semi_]auto_parallel. [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.954.971 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bprop_depend_298] Added global python symbol: {C : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.959.925 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bprop_load_299] Added global python symbol: {C : } [INFO] OPTIMIZER(164040,ffffb1c2c440,python):2024-01-10-11:37:31.968.914 [mindspore/ccsrc/frontend/optimizer/ad/bprop_utils.cc:70] GetBprop] Fail to find bprop function for UpdateState. fn: None [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:31.985.370 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractUMonad(ValueAny)}, g: _zeros_like_u_monad_302 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.000.108 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractUMonad(ValueAny), AbstractUMonad(ValueAny)}, g: hyper_map_303 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.002.709 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractUMonad(ValueAny), AbstractUMonad(ValueAny)}, g: _add_umonad_umonad_304 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.005.540 [mindspore/ccsrc/pipeline/jit/ps/action.cc:282] AbstractAnalyze] function call depth: 0, simulate call depth: 0 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:32.005.760 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:222] Run] Specialize set top func graph context: {FuncGraph: 1_mindspore_train_dataset_helper__DataWrapper_construct_300 Args: [0]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [1]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [2]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [3]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [4]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [5]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [6]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [7]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [8]: AbstractRefTensor(key: global_step ref_value: AbstractRefTensor(shape: (1), element: AbstractScalar(Type: Int32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [9]: AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [10]: AbstractRefTensor(key: moments.conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [11]: AbstractRefTensor(key: moments.fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [12]: AbstractRefTensor(key: moments.fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [13]: AbstractRefTensor(key: moments.fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [14]: AbstractRefTensor(key: moments.fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [15]: AbstractRefTensor(key: moments.fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [16]: AbstractRefTensor(key: moments.fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [17]: AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [18]: AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), Parent: } [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.051.055 [mindspore/ccsrc/frontend/parallel/pynative_shard/pynative_shard.cc:334] PynativeShard] Only auto_parallel and semi_auto_parallel support pynative shard [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.057.219 [mindspore/ccsrc/pipeline/jit/ps/action.cc:282] AbstractAnalyze] function call depth: 0, simulate call depth: 0 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:37:32.057.678 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:222] Run] Specialize set top func graph context: {FuncGraph: 251_1_mindspore_train_dataset_helper__DataWrapper_construct_315 Args: [0]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [1]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [2]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [3]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [4]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [5]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [6]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [7]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [8]: AbstractRefTensor(key: global_step ref_value: AbstractRefTensor(shape: (1), element: AbstractScalar(Type: Int32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [9]: AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [10]: AbstractRefTensor(key: moments.conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [11]: AbstractRefTensor(key: moments.fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [12]: AbstractRefTensor(key: moments.fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [13]: AbstractRefTensor(key: moments.fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [14]: AbstractRefTensor(key: moments.fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [15]: AbstractRefTensor(key: moments.fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [16]: AbstractRefTensor(key: moments.fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [17]: AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [18]: AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), Parent: } [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.064.780 [mindspore/ccsrc/frontend/parallel/pynative_shard/pynative_shard.cc:334] PynativeShard] Only auto_parallel and semi_auto_parallel support pynative shard [INFO] OPTIMIZER(164040,ffffb1c2c440,python):2024-01-10-11:37:32.070.417 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] OPTIMIZER(164040,ffffb1c2c440,python):2024-01-10-11:37:32.071.509 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] OPTIMIZER(164040,ffffb1c2c440,python):2024-01-10-11:37:32.072.358 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.072.517 [mindspore/ccsrc/frontend/parallel/cache_embedding/cache_embedding.cc:702] AddCacheEmbedding] Parameters are all not cache enable. [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.073.567 [mindspore/ccsrc/frontend/parallel/pass/assign_add_opt.cc:120] AssignAddOpt] parallel::g_device_manager is not initialized. [INFO] OPTIMIZER(164040,ffffb1c2c440,python):2024-01-10-11:37:32.073.633 [mindspore/ccsrc/frontend/optimizer/comm_op_reuse_tag.cc:59] AddCommOpReuseTag] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.073.656 [mindspore/ccsrc/frontend/parallel/pass/overlap_opt_shard_in_pipeline.cc:70] OverlapOptShardInPipeline] parallel::g_device_manager is not initialized. [INFO] OPTIMIZER(164040,ffffb1c2c440,python):2024-01-10-11:37:32.073.678 [mindspore/ccsrc/frontend/optimizer/grouped_pairwise_exchange_alltoall.cc:673] SetGroupedPairwiseExchangeAllToAll] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.073.742 [mindspore/ccsrc/frontend/parallel/pass/overlap_gradmatmul_and_gradallreduce.cc:358] OverlapGradMatmulAndGradAllreduce] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.073.763 [mindspore/ccsrc/frontend/parallel/pass/split_matmul_comm_elementwise_fp.cc:184] SplitMatmulCommElementwiseFp] SplitMatmulCommElementwiseFp is only support under [semi_]auto_parallel, skip it. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.073.794 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end optimize action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.073.816 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start auto_monad_reorder action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.074.176 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end auto_monad_reorder action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.074.211 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start get_jit_bprop_graph action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.074.226 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end get_jit_bprop_graph action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.074.245 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start eliminate_special_op_node action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.075.128 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end eliminate_special_op_node action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.075.177 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start validate action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.075.458 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end validate action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.075.502 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start distribtued_split action. [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.075.535 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:372] GenerateStrategy] Current parallel mode is data_parallel [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.075.548 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:384] GenerateStrategy] Generated distributed strategy is 1 [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.075.942 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:1270] Run] All nodes are on this precoess so there's no need to build and split distributed graph. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.075.976 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end distribtued_split action. [INFO] PROFILER(164040,ffffb1c2c440,python):2024-01-10-11:37:32.076.055 [mindspore/ccsrc/plugin/device/ascend/hal/profiler/parallel_strategy_profiling.cc:48] IsProfilingParallelStrategyEnabled] Profiling parallel strategy is disabled. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.076.077 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start task_emit action. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.076.496 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.076.523 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.076.536 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1252] SetRunMode] Run graph mode with kernel by kernel by configuration. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:37:32.076.605 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:546] CompileGraphs] Status record: start compile function graph: 381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.076.799 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unify_mindir_pm_0_erase_invalid_micro_depend in 2.32 us [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.077.941 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.077.975 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.341 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.363 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.390 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.404 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.423 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.435 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.451 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.462 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.489 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.501 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.517 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.528 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.543 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.554 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.569 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.580 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.595 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.606 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.622 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.634 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.648 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.660 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.686 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.701 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.718 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.730 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.745 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.757 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.772 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.784 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.799 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.818 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.834 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.846 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.862 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.873 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.889 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.901 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.916 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.927 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.943 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.954 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.969 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.980 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.078.995 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.007 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.022 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.033 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.048 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.059 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.074 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.086 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.102 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.113 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.134 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.146 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.164 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.175 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.192 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.205 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.220 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.231 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.247 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.258 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.274 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.285 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.300 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.311 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.327 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.338 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.354 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.365 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.381 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.392 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.408 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.419 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.434 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.445 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.468 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.479 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.494 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.505 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.520 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.531 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.546 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.557 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.573 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.584 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.599 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.610 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.625 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.636 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.649 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.660 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.675 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.686 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.701 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.713 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.727 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.739 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.754 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.770 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.785 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.796 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.811 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.822 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.837 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.848 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.865 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.877 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.892 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.904 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.919 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.930 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.945 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.956 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.971 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.982 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.079.996 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.007 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.021 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.032 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.047 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.058 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.078 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.090 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.106 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.117 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.132 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.143 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.160 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.171 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.186 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.197 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.212 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.223 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.237 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.248 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.263 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.274 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.289 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.300 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.314 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.325 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.340 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.351 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.366 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.377 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.396 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.408 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.423 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.434 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.450 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.461 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.476 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.487 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.502 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.513 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.528 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.540 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.555 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.566 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.581 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.593 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.608 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.619 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.633 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.643 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.657 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.668 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.683 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.699 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.714 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.726 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.741 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.752 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.767 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.778 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.793 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.804 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.819 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.830 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.844 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.855 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.870 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.881 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.895 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.906 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.920 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.931 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.945 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.956 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.970 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.080.981 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:37:32.081.061 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:727] CompileGraph] Compile graph: 381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316, Split segments size: 2 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:37:32.081.109 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:762] CompileGraphFromSegment] Compile normal segment, the first node: @381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316:equiv_CNode_317{[0]: ValueNode Load, [1]: param_fc3.bias, [2]: ValueNode U} [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.081.558 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:421] CompileGraph] Status record: start compile graph. [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.081.603 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:1940] ConstructKernelGraph] Create graph: 1 [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.082.399 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:872] CreateNewValueNode] Data sync for Tensor Tensor(shape=[1], dtype=Int32, value=[1]) [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.082.595 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:872] CreateNewValueNode] Data sync for Tensor Tensor(shape=[], dtype=Float32, value=0.25) [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.084.001 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:2697] ConstructOutput] Output:@381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316:CNode_278{[0]: ValueNode Depend, [1]: CNode_278, [2]: CNode_318} [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.085.881 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:193] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:1 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.086.346 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 88.93 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.086.937 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 539.85 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.087.846 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:154] CommonUnifyMindIR] start common unify mindir opt graph:1 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.088.173 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.088.592 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 414.91 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.088.626 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.088.668 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 38.43 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.088.688 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.088.705 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 14.92 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.088.719 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.089.275 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 538.92 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.090.249 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:79] BackendCommonOptimization] Status record: start common optimization. graph id: 1 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.090.580 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: add_dynamic_shape_attr [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.091.365 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_0_add_dynamic_shape_attr in 777.42 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.091.402 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.091.457 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_1_convert_dynamic_broadcast_to in 50.51 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.092.045 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_2_convert_const_input_to_attr in 554.49 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.092.552 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_3_custom_op_const_input_to_attr in 467.14 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.092.984 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 392.98 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.094.411 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 1380.29 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.094.471 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 16.17 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.094.859 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_7_flatten_concat_fission in 355.35 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.095.222 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 327.55 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.095.584 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_9_broadcast_to_fusion in 325.75 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.096.085 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 466.32 us [WARNING] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.097.007 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:92] BackendCommonOptimization] [PROF]backend_common_optimization costs 6757 usec. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.097.041 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:99] BackendCommonOptimization] Status record: end common optimization. graph id: 1 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.097.823 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_0_renorm_split in 373.15 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.097.861 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.098.782 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_1_reduce_axis_update in 909.53 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.098.826 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim HistogramFixedWidth not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.098.847 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_2_histogram_fixed_width_fusion in 20.35 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.098.863 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim ClipByNorm not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.098.877 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_3_clip_by_norm_fission in 13.45 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.098.892 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.098.905 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_4_tensor_arry_add_flow_cond1 in 12.67 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.098.919 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayGather not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.098.932 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_5_tensor_arry_add_flow_cond2 in 11.8 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.098.945 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.098.959 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_6_ge_tensor_arry_cast_index in 11.56 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.098.972 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArray not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.098.986 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_7_tensor_array_prepare in 11.85 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.098.998 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.099.038 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_8_space_to_batch_nd_attr_update in 35.84 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.099.054 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.099.083 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_9_batch_to_space_nd_attr_update in 26.57 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.099.098 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: avg_pool_grad_unify_mindir [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.099.134 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_10_avg_pool_grad_unify_mindir in 32.65 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.099.512 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_11_adam_weight_decay_unify_mindir in 350.49 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.099.877 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_12_cdist_fission in 330.18 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.100.237 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_13_cdist_grad_fission in 324.28 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.100.727 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_14_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 440.53 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.101.996 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_15_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 1219.99 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.102.431 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_16_sparse_softmax_cross_entropy_with_logits_unify_mindir in 382.7 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.102.823 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_17_dropout_unify_mindir1 in 349.41 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.102.853 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.228 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_18_dropoutgrad_unify_mindir in 362.94 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.258 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.276 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_19_neighbor_exchange_unify_mindir in 18.5 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.292 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.306 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_20_neighbor_exchange_v2_unify_mindir in 12.83 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.319 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.333 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_21_neighbor_exchange_v2_grad_unify_mindir in 11.84 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.346 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.359 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_22_all_to_all_unify_mindir in 11.65 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.374 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim QuantDTypeCast not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.390 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_23_quant_dtype_cst_adjust in 16.2 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.404 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim FSEDecode not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.418 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_24_fse_decode_adjust in 13.16 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.431 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.455 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_25_bn_split in 21.66 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.468 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.534 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_26_bn_grad_unify_mindir in 60.42 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.554 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.569 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_27_bn_grad_split in 14.31 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.582 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.596 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_28_batchnorm_to_bninfer in 11.87 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.609 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.622 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_29_batchnormgrad_to_bninfergrad in 11.76 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.635 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.103.648 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_30_batch_norm_grad_infer_fission in 11.35 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.104.523 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_31_ascend_mindir_op_adapter in 841.91 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.104.563 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_32_DropoutGenMask in 1.54 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.104.963 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_33_lamb_fission_ge in 369.61 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.105.892 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_34_print_insert_placeholder_for_tensor_name in 890.85 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.106.318 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_35_getnext_for_ge in 382.93 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.106.704 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_36_sync_bn_split in 346.46 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.107.083 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_37_sync_bn_grad_split in 340.77 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.107.459 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_38_adaptive_max_pool2d_ge_fusion in 339.37 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.107.836 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_39_avg_pool_grad_for_ge in 340.08 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.107.865 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:260] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.108.824 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_40_FlashAttentionFusionV1 in 948.64 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.108.860 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:374] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.109.847 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_41_FlashAttentionFusionV1 in 978.66 us [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.112.002 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.112.037 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.112.055 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:132] GetRunMode] RunMode::kKernelMode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.113.377 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 1268.6 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.115.305 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_0_seed_adapter in 1198.71 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.115.348 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: env_op_attr_update [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.116.128 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_1_env_op_attr_update in 765.36 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.116.543 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_2_process call inline in 366.37 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.116.866 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_3_expander_fallback in 274.46 us [WARNING] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.117.521 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:139] GEBackendOptimizeACL] [PROF]ascend_backend_optimize_acl costs 3432 usec. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.117.583 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] GetNext is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.118.114 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2D is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.118.396 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPool is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.118.518 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2D is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.118.677 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPool is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.118.864 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.119.260 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.119.482 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.119.710 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] SoftmaxCrossEntropyWithLogits is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.120.194 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.120.331 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.120.461 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.120.716 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.120.998 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.121.101 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.121.198 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.121.425 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.121.660 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.121.801 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.121.920 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.122.142 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.122.373 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPoolGrad is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.122.609 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AssignAdd is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.122.764 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2DBackpropInput is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.122.938 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2DBackpropFilter is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.123.106 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.123.315 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.123.482 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPoolGrad is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.123.665 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2DBackpropFilter is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.123.798 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.123.976 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.124.096 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.124.210 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.124.322 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.124.427 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.124.532 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.124.696 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.125.304 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 168.04 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.126.319 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 966.18 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.127.218 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 852.48 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.128.833 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 1561.48 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.130.398 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_4_insert_type_transform_op in 1508.31 us [WARNING] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.131.064 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:179] GEBackendOptimizeACLAfterKernelSelect] [PROF]ascend_backend_optimize_acl_after_kernel_select costs 5976 usec. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.131.183 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_0_DropoutGenMask in 0.9 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.131.546 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_1_cse in 317 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.132.735 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_redundant_op in 1143.98 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.132.961 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_3_eliminate_maketuple_getitem in 167 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.132.995 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_4_MergeTransData in 1.78 us [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.134.096 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive GetNext [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.134.340 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:467] ConvertAny] Value: ValueTuple [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.134.438 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive GetNext [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.134.470 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.134.587 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.134.617 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive OneHot [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.134.781 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive OneHot [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.134.811 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2D [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.134.888 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.134.928 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.134.959 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.135.043 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2D [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.135.068 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.135.143 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.135.165 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPool [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.135.277 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPool [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.135.302 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2D [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.135.358 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.135.407 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.135.434 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.135.509 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2D [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.135.532 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.135.603 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.135.626 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPool [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.135.707 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPool [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.135.731 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Flatten [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.135.818 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Flatten [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.135.842 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.135.954 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.135.980 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.136.144 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.136.174 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.136.248 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.136.271 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.136.364 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.136.390 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.136.492 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.136.517 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.136.600 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.136.622 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.136.708 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.136.734 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.136.833 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.136.860 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.136.944 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.136.969 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive SoftmaxCrossEntropyWithLogits [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.137.078 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive SoftmaxCrossEntropyWithLogits [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.137.105 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.137.188 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.137.214 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.137.293 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.137.318 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReduceMean [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.137.448 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReduceMean [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.137.478 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.137.584 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.137.609 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.137.763 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.137.803 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.137.951 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.138.070 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.138.103 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.138.165 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.138.253 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.138.280 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReluGrad [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.138.355 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReluGrad [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.138.381 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.138.466 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.138.492 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.138.573 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.138.599 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.138.657 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.138.733 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.138.755 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.138.842 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.138.867 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.138.923 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.138.996 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.139.034 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReluGrad [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.139.116 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReluGrad [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.139.138 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.139.221 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.139.245 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.139.338 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.139.366 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.139.429 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.139.510 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.139.537 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.139.621 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.139.645 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.139.704 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.139.781 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.139.806 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.139.900 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.139.926 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPoolGrad [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.140.058 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPoolGrad [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.140.087 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReluGrad [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.140.167 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReluGrad [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.140.201 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive AssignAdd [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.140.283 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive AssignAdd [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.140.306 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2DBackpropInput [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.140.387 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.140.418 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.140.505 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2DBackpropInput [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.140.533 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2DBackpropFilter [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.140.598 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.140.635 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.140.664 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.140.750 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2DBackpropFilter [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.140.779 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.140.847 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.140.931 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.140.958 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.141.085 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.141.110 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPoolGrad [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.141.221 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPoolGrad [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.141.248 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReluGrad [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.141.328 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReluGrad [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.141.366 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2DBackpropFilter [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.141.428 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.141.463 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.141.488 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.141.575 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2DBackpropFilter [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.141.599 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.141.662 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.141.788 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.141.816 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.141.925 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.141.952 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.142.052 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.142.076 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.142.172 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.142.197 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.142.298 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.142.325 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.142.422 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.142.449 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.142.544 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.142.583 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.142.655 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.142.677 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.142.771 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [WARNING] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.142.791 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:262] CreateKernel] [PROF]create_kernel costs 8738 usec. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.143.050 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/Reshape-op0, index: 0 to input Default/GetNext-op1, index: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.143.090 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1, index: 0 to input Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2, index: 0 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.143.116 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/Reshape-op1, index: 0 to input Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1, index: 0 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.143.140 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/Reshape-op2, index: 0 to input Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/Mul-op0, index: 0 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.143.164 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/gradFlatten-expand/Reshape-op1, index: 0 to input Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op6, index: 0 [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.143.181 [mindspore/ccsrc/backend/common/session/session_basic.cc:1140] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 1 start [INFO] DEBUG(164040,ffffb1c2c440,python):2024-01-10-11:37:32.143.194 [mindspore/ccsrc/debug/summary/summary.cc:33] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 1 start [INFO] DEBUG(164040,ffffb1c2c440,python):2024-01-10-11:37:32.143.209 [mindspore/ccsrc/debug/summary/summary.cc:54] RecurseSetSummaryNodesForAllGraphs] The total summary nodes is: 0 for graph: 1 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.143.841 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8's input.Send node Default/StreamSend-op0 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op7, recv node Default/StreamRecv-op0 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.143.928 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8's output.Send node Default/StreamSend-op1 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8, recv node Default/StreamRecv-op1 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op8 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.143.988 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9's input.Send node Default/StreamSend-op2 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op3, recv node Default/StreamRecv-op2 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.144.038 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9's output.Send node Default/StreamSend-op3 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9, recv node Default/StreamRecv-op3 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op9 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.144.098 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10's input.Send node Default/StreamSend-op4 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op8, recv node Default/StreamRecv-op4 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.144.152 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10's output.Send node Default/StreamSend-op5 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10, recv node Default/StreamRecv-op5 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op10 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.144.214 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11's input.Send node Default/StreamSend-op6 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op4, recv node Default/StreamRecv-op6 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.144.267 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11's output.Send node Default/StreamSend-op7 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11, recv node Default/StreamRecv-op7 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op11 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.144.326 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12's input.Send node Default/StreamSend-op8 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op9, recv node Default/StreamRecv-op8 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.144.388 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12's output.Send node Default/StreamSend-op9 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12, recv node Default/StreamRecv-op9 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op12 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.144.443 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13's input.Send node Default/StreamSend-op10 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op5, recv node Default/StreamRecv-op10 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.144.495 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13's output.Send node Default/StreamSend-op11 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13, recv node Default/StreamRecv-op11 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op13 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.144.565 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14's input.Send node Default/StreamSend-op12 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/gradConv2D-expand/Conv2DBackpropFilter-op1, recv node Default/StreamRecv-op12 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.144.617 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14's output.Send node Default/StreamSend-op13 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14, recv node Default/StreamRecv-op13 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op14 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.144.678 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15's input.Send node Default/StreamSend-op14 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv1-Conv2d/gradConv2D-expand/Conv2DBackpropFilter-op1, recv node Default/StreamRecv-op14 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.144.730 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15's output.Send node Default/StreamSend-op15 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15, recv node Default/StreamRecv-op15 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op15 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.144.871 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.144.939 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:0 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.144.969 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.144.992 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:0 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.019 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.041 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:1 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.067 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.087 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:2 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.110 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.131 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:2 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.153 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.171 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:3 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.191 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.210 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:1 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.235 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.255 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:4 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.279 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.299 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:4 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.322 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.341 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:5 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.363 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.381 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:3 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.405 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.439 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:6 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.463 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.483 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:6 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.506 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.525 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:7 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.545 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.564 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:5 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.588 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.608 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:8 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.629 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.650 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:8 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.672 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.750 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:9 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.773 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.793 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:7 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.816 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.836 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:10 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.858 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.876 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:10 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.900 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.920 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:11 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.943 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.964 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:9 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.145.989 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.019 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:12 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.041 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.061 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:12 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.083 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.101 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:13 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.122 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.140 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:11 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.164 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.182 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:14 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.203 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.220 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:14 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.242 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.260 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:15 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.280 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.297 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:13 [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.322 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.341 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:15 [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.363 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:891] PrintGraphExecuteOrder] Graph 1 execution order: [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.411 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[0], node name[Default/GetNext-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:outputs{[0]: ValueNode GetNext}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.471 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[1], node name[Default/Reshape-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_Reshape, [1]: CNode_278, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[32])}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.540 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[2], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/OneHot-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_OneHot, [1]: 319, [2]: ValueNode Tensor(shape=[], dtype=Int64, value=10), [3]: ValueNode Tensor(shape=[], dtype=Float32, value=1), [4]: ValueNode Tensor(shape=[], dtype=Float32, value=0), [5]: ValueNode -1}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.611 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[3], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv1-Conv2d/Conv2D-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode Conv2D, [1]: CNode_278, [2]: equiv_CNode_320}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.663 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[4], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op4], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_ReLU, [1]: equiv_CNode_278}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.715 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[5], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op3], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode MaxPool, [1]: equiv_CNode_278}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.772 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[6], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/Conv2D-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode Conv2D, [1]: equiv_CNode_278, [2]: equiv_CNode_321}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.820 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[7], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op5], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_ReLU, [1]: equiv_CNode_278}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.866 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[8], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode MaxPool, [1]: equiv_CNode_278}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.922 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[9], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_Flatten, [1]: equiv_CNode_278}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.146.984 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[10], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/MatMul-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode MatMul, [1]: equiv_CNode_278, [2]: equiv_CNode_322}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.043 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[11], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/BiasAdd-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_BiasAdd, [1]: equiv_CNode_278, [2]: equiv_CNode_323, [3]: ValueNode 0}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.099 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[12], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op6], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_ReLU, [1]: equiv_CNode_278}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.153 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[13], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/MatMul-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode MatMul, [1]: equiv_CNode_278, [2]: equiv_CNode_324}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.211 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[14], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/BiasAdd-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_BiasAdd, [1]: equiv_CNode_278, [2]: equiv_CNode_325, [3]: ValueNode 0}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.258 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[15], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op7], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_ReLU, [1]: equiv_CNode_278}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.308 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[16], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/MatMul-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode MatMul, [1]: equiv_CNode_278, [2]: equiv_CNode_326}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.359 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[17], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_BiasAdd, [1]: equiv_CNode_278, [2]: equiv_CNode_317, [3]: ValueNode 0}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.411 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[18], node name[Default/Reshape-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_Reshape, [1]: equiv_CNode_278, [2]: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10])}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.449 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[19], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/SoftmaxCrossEntropyWithLogits-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode SoftmaxCrossEntropyWithLogits, [1]: 319, [2]: 319}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.485 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[20], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/Mul-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_Mul, [1]: 319, [2]: ValueNode Tensor(shape=[32, 1], dtype=Float32, value=[...])}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.531 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[21], node name[Default/Reshape-op2], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_Reshape, [1]: 319, [2]: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10])}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.575 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[22], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/ReduceMean-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_ReduceMean, [1]: 319, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), [3]: ValueNode false}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.614 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[23], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op10], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 319, [2]: equiv_CNode_326}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.659 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[24], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op7], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 319, [2]: equiv_CNode_278}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.691 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[25], node name[Default/StreamSend-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_328{[0]: ValueNode StreamSend}], event id[0] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.717 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[26], node name[Default/StreamRecv-op0], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_329{[0]: ValueNode StreamRecv}], event id[0] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.759 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[27], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 327}], group[hccl_world_group] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.787 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[28], node name[Default/StreamSend-op1], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_330{[0]: ValueNode StreamSend}], event id[1] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.821 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[29], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op3], logic id[4294967295], stream id[0], node info[@kernel_graph_1:331{[0]: ValueNode PrimFunc_BiasAddGrad, [1]: 319, [2]: ValueNode 0}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.848 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[30], node name[Default/StreamSend-op2], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_332{[0]: ValueNode StreamSend}], event id[2] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.872 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[31], node name[Default/StreamRecv-op2], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_333{[0]: ValueNode StreamRecv}], event id[2] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.918 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[32], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 331}], group[hccl_world_group] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.944 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[33], node name[Default/StreamSend-op3], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_334{[0]: ValueNode StreamSend}], event id[3] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.147.968 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[34], node name[Default/StreamRecv-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_335{[0]: ValueNode StreamRecv}], event id[1] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.023 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[35], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op8], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.070 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[36], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/gradReLU-expand/ReluGrad-op4], logic id[4294967295], stream id[0], node info[@kernel_graph_1:336{[0]: ValueNode PrimFunc_ReluGrad, [1]: 327, [2]: equiv_CNode_278}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.109 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[37], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op11], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 336, [2]: equiv_CNode_324}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.148 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[38], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op8], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 336, [2]: equiv_CNode_278}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.173 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[39], node name[Default/StreamSend-op4], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_337{[0]: ValueNode StreamSend}], event id[4] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.197 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[40], node name[Default/StreamRecv-op4], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_338{[0]: ValueNode StreamRecv}], event id[4] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.235 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[41], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 327}], group[hccl_world_group] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.266 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[42], node name[Default/StreamSend-op5], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_339{[0]: ValueNode StreamSend}], event id[5] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.292 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[43], node name[Default/StreamRecv-op3], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_340{[0]: ValueNode StreamRecv}], event id[3] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.345 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[44], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op9], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.378 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[45], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op4], logic id[4294967295], stream id[0], node info[@kernel_graph_1:331{[0]: ValueNode PrimFunc_BiasAddGrad, [1]: 336, [2]: ValueNode 0}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.403 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[46], node name[Default/StreamSend-op6], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_341{[0]: ValueNode StreamSend}], event id[6] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.431 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[47], node name[Default/StreamRecv-op6], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_342{[0]: ValueNode StreamRecv}], event id[6] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.470 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[48], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 331}], group[hccl_world_group] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.495 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[49], node name[Default/StreamSend-op7], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_343{[0]: ValueNode StreamSend}], event id[7] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.519 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[50], node name[Default/StreamRecv-op5], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_344{[0]: ValueNode StreamRecv}], event id[5] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.568 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[51], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op10], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.613 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[52], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/gradReLU-expand/ReluGrad-op5], logic id[4294967295], stream id[0], node info[@kernel_graph_1:336{[0]: ValueNode PrimFunc_ReluGrad, [1]: 327, [2]: equiv_CNode_278}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.659 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[53], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op6], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 336, [2]: equiv_CNode_322}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.709 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[54], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op9], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 336, [2]: equiv_CNode_278}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.734 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[55], node name[Default/StreamSend-op8], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_345{[0]: ValueNode StreamSend}], event id[8] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.759 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[56], node name[Default/StreamRecv-op8], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_346{[0]: ValueNode StreamRecv}], event id[8] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.799 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[57], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 327}], group[hccl_world_group] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.823 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[58], node name[Default/StreamSend-op9], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_347{[0]: ValueNode StreamSend}], event id[9] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.847 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[59], node name[Default/StreamRecv-op7], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_348{[0]: ValueNode StreamRecv}], event id[7] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.898 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[60], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op11], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.930 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[61], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op5], logic id[4294967295], stream id[0], node info[@kernel_graph_1:331{[0]: ValueNode PrimFunc_BiasAddGrad, [1]: 336, [2]: ValueNode 0}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.954 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[62], node name[Default/StreamSend-op10], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_349{[0]: ValueNode StreamSend}], event id[10] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.148.977 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[63], node name[Default/StreamRecv-op10], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_350{[0]: ValueNode StreamRecv}], event id[10] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.021 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[64], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 331}], group[hccl_world_group] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.045 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[65], node name[Default/StreamSend-op11], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_351{[0]: ValueNode StreamSend}], event id[11] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.069 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[66], node name[Default/StreamRecv-op9], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_352{[0]: ValueNode StreamRecv}], event id[9] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.120 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[67], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op12], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.157 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[68], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/gradFlatten-expand/Reshape-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:353{[0]: ValueNode PrimFunc_Reshape, [1]: 327, [2]: ValueNode (32, 16, 5, 5)}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.218 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[69], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/gradMaxPool-expand/MaxPoolGrad-op2], logic id[4294967295], stream id[0], node info[@kernel_graph_1:354{[0]: ValueNode MaxPoolGrad, [1]: equiv_CNode_278, [2]: equiv_CNode_278, [3]: 353}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.260 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[70], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/gradReLU-expand/ReluGrad-op6], logic id[4294967295], stream id[0], node info[@kernel_graph_1:336{[0]: ValueNode PrimFunc_ReluGrad, [1]: 354, [2]: equiv_CNode_278}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.311 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[71], node name[Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AssignAdd, [1]: param_global_step, [2]: ValueNode Tensor(shape=[1], dtype=Int32, value=[1]), [3]: CNode_355}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.355 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[72], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/gradConv2D-expand/Conv2DBackpropInput-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:356{[0]: ValueNode Conv2DBackpropInput, [1]: 336, [2]: equiv_CNode_321, [3]: ValueNode (32, 6, 14, 14)}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.403 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[73], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/gradConv2D-expand/Conv2DBackpropFilter-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:356{[0]: ValueNode Conv2DBackpropFilter, [1]: 336, [2]: equiv_CNode_278, [3]: ValueNode (16, 6, 5, 5)}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.435 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[74], node name[Default/StreamSend-op12], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_357{[0]: ValueNode StreamSend}], event id[12] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.460 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[75], node name[Default/StreamRecv-op12], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_358{[0]: ValueNode StreamRecv}], event id[12] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.500 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[76], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 356}], group[hccl_world_group] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.526 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[77], node name[Default/StreamSend-op13], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_359{[0]: ValueNode StreamSend}], event id[13] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.550 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[78], node name[Default/StreamRecv-op11], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_360{[0]: ValueNode StreamRecv}], event id[11] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.602 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[79], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op13], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.677 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[80], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc3.bias, [2]: param_moments.fc3.bias, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_361}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.758 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[81], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/gradMaxPool-expand/MaxPoolGrad-op3], logic id[4294967295], stream id[0], node info[@kernel_graph_1:354{[0]: ValueNode MaxPoolGrad, [1]: equiv_CNode_278, [2]: equiv_CNode_278, [3]: 356}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.804 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[82], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/gradReLU-expand/ReluGrad-op7], logic id[4294967295], stream id[0], node info[@kernel_graph_1:336{[0]: ValueNode PrimFunc_ReluGrad, [1]: 354, [2]: equiv_CNode_278}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.846 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[83], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv1-Conv2d/gradConv2D-expand/Conv2DBackpropFilter-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:362{[0]: ValueNode Conv2DBackpropFilter, [1]: 336, [2]: CNode_278, [3]: ValueNode (6, 1, 5, 5)}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.880 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[84], node name[Default/StreamSend-op14], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_363{[0]: ValueNode StreamSend}], event id[14] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.907 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[85], node name[Default/StreamRecv-op14], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_364{[0]: ValueNode StreamRecv}], event id[14] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.950 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[86], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 362}], group[hccl_world_group] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.149.979 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[87], node name[Default/StreamSend-op15], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_365{[0]: ValueNode StreamSend}], event id[15] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.150.005 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[88], node name[Default/StreamRecv-op13], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_366{[0]: ValueNode StreamRecv}], event id[13] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.150.061 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[89], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op14], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.150.138 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[90], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc3.weight, [2]: param_moments.fc3.weight, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_367}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.150.209 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[91], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc2.bias, [2]: param_moments.fc2.bias, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_368}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.150.280 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[92], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc2.weight, [2]: param_moments.fc2.weight, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_369}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.150.362 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[93], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc1.bias, [2]: param_moments.fc1.bias, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_370}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.150.432 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[94], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc1.weight, [2]: param_moments.fc1.weight, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_371}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.150.503 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[95], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_conv2.weight, [2]: param_moments.conv2.weight, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_372}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.150.529 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[96], node name[Default/StreamRecv-op15], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_373{[0]: ValueNode StreamRecv}], event id[15] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.150.582 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[97], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op15], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.150.652 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[98], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_conv1.weight, [2]: param_moments.conv1.weight, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_374}] [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.151.166 [mindspore/ccsrc/backend/common/somas/somas.cc:143] Assign] Start Somas Assign for graph 1 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.151.609 [mindspore/ccsrc/backend/common/somas/somas.cc:151] Assign] Somas Configure success, configuration info: Device Name: Ascend Run by execution order: 1 Enable debug log: 0 Debug log path: . [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.151.631 [mindspore/ccsrc/backend/common/somas/somas.cc:154] Assign] Start Initialize SOMAS Model [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.152.993 [mindspore/ccsrc/backend/common/somas/somas.cc:1065] GraphOutputProcess] Set 1 graph output tensors' aligned size to 0. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.153.631 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 1 output 2 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.153.669 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 9 output 10 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.153.705 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 18 output 19 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.153.730 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 22 output 23 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.153.790 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 40 output 47 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.153.829 [mindspore/ccsrc/backend/common/somas/somas.cc:1135] RefNodeProcess] Special Tensor total size: RefNode: input 107008 output 351828 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.153.895 [mindspore/ccsrc/backend/common/somas/somas.cc:555] InitSomasModel] Created 2 streams (0 groups), 99 nodes, 69 tensors, 5 union tensors lists, and 0 contiguous tensors lists [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.154.490 [mindspore/ccsrc/backend/common/somas/somas.cc:157] Assign] End Initialize SOMAS Model [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.154.511 [mindspore/ccsrc/backend/common/somas/somas.cc:176] Assign] Start Computing Conflict Matrix [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.154.524 [mindspore/ccsrc/backend/common/somas/somas.cc:1286] ComputeBasicMatrix] Start Conflict Computing (Bitset Model) [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.154.546 [mindspore/ccsrc/backend/common/somas/somas.cc:1291] ComputeBasicMatrix] Start Bitset [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.154.586 [mindspore/ccsrc/backend/common/somas/somas.cc:1299] ComputeBasicMatrix] Start Path Computing [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.154.611 [mindspore/ccsrc/backend/common/somas/somas.cc:1307] ComputeBasicMatrix] End Path Computing [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.154.622 [mindspore/ccsrc/backend/common/somas/somas.cc:1309] ComputeBasicMatrix] Start Tensor Relation Computing [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.154.698 [mindspore/ccsrc/backend/common/somas/somas.cc:1462] ComputeMultiTensorConflicts] Start Computing Conflicts Pairs, tensors list size is 60 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.154.776 [mindspore/ccsrc/backend/common/somas/somas.cc:1469] ComputeMultiTensorConflicts] End Computing Conflicts Pairs (time taken 0ms) [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.154.791 [mindspore/ccsrc/backend/common/somas/somas.cc:1367] ComputeBasicMatrix] End Basic Conflict Computing (Bitset Model)(time taken 0ms) [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.154.826 [mindspore/ccsrc/backend/common/somas/somas.cc:178] Assign] End Computing Conflict Matrix [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.154.838 [mindspore/ccsrc/backend/common/somas/somas.cc:1533] Solve] Somas Assign start... [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.154.871 [mindspore/ccsrc/backend/common/somas/somas.cc:1555] Solve] Start Solving [INFO] PRE_ACT(164040,fffeb6ffd0f0,python):2024-01-10-11:37:32.155.143 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 55 tensors [INFO] PRE_ACT(164040,fffeb77fe0f0,python):2024-01-10-11:37:32.155.155 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 55 tensors [INFO] PRE_ACT(164040,fffeb7fff0f0,python):2024-01-10-11:37:32.155.148 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 55 tensors [INFO] PRE_ACT(164040,fffeb67fc0f0,python):2024-01-10-11:37:32.155.168 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 55 tensors [INFO] PRE_ACT(164040,fffeb77fe0f0,python):2024-01-10-11:37:32.155.308 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 3/4 2196992 Bytes (0.00204611 GB) Single Object size(>), index(<) bestfit [INFO] PRE_ACT(164040,fffeb6ffd0f0,python):2024-01-10-11:37:32.155.317 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 2/4 2196992 Bytes (0.00204611 GB) Shared Objects size(>), index(<) smallest [INFO] PRE_ACT(164040,fffeb7fff0f0,python):2024-01-10-11:37:32.155.336 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 1/4 2196992 Bytes (0.00204611 GB) Shared Objects size(>), index(<) bestfit [INFO] PRE_ACT(164040,fffeb67fc0f0,python):2024-01-10-11:37:32.155.344 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 4/4 2196992 Bytes (0.00204611 GB) Single Object size(>), index(<) smallest [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.385 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:176] Solving] SOMAS SOLVER RESUME: [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.399 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:177] Solving] Best Solution:[1/4] [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.415 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:178] Solving] Best result:2196992 Bytes 0.00204611 GB (0.00204611 GB + 0 GB from lifelong tensors) [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.427 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:181] Solving] Best timing:0 ms [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.438 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:182] Solving] Best algorithm: Shared Objects [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.448 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:183] Solving] Best sorting strategy: size(>), index(<) [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.459 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:184] Solving] Best offset strategy: bestfit [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.469 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:185] Solving] Time elapsed: 0 ms [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.481 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:186] Solving] Spread:0 %% [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.550 [mindspore/ccsrc/backend/common/somas/somas.cc:1564] Solve] End Solving [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.630 [mindspore/ccsrc/backend/common/somas/somas.cc:2096] GenGraphStatisticInfo] Lower Bound: 2186752 (0.00203657 GB), Upper Bound: 4660224 (0.00434017 GB) [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.643 [mindspore/ccsrc/backend/common/somas/somas.cc:2099] GenGraphStatisticInfo] Total Dynamic Size (Upper Bound): 4660224 Theoretical Optimal Size (Lower Bound): 2186752 Total Workspace Size: 0 Total Communication Input Tensor Size: 248832 Total Communication Output Tensor Size: 0 Total LifeLong All Tensor Size: 0 Total LifeLong Start Tensor Size: 0 Total LifeLong End Tensor Size: 512 Reused Size(Allocate Size): 0 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.656 [mindspore/ccsrc/backend/common/somas/somas.cc:1583] Solve] Somas Assign end. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.799 [mindspore/ccsrc/backend/common/somas/somas.cc:380] UpdateSomasResultToGraph] Merged Block size: 12 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.815 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 0, offset: 1205248, size: 602624 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.826 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 1, offset: 602624, size: 602624 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.837 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 2, offset: 0, size: 602624 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.847 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 3, offset: 1807872, size: 192512 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.858 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 4, offset: 2000384, size: 131584 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.878 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 5, offset: 2131968, size: 40448 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.889 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 6, offset: 2182144, size: 9728 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.900 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 7, offset: 2172416, size: 9728 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.910 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 8, offset: 2191872, size: 3584 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.920 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 9, offset: 2196480, size: 512 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.932 [mindspore/ccsrc/backend/common/somas/somas.cc:189] Assign] Somas Allocate end. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.155.944 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:310] DoSomas] Somas allocate success for graph 1 somas size: 2196992 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.156.151 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:801] CreateDeviceAddress] Status record: start create device address. graph id: 1 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.156.945 [mindspore/ccsrc/runtime/device/device_address_utils.cc:454] CreateValueNodeDeviceAddress] No device address for value node:Default/data-17, debug name:ValueNode U [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.455 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2, index is 0; cur kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.506 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2, index is 0; cur kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.533 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is conv2.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.554 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is conv2.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.570 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc1.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.587 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc1.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.602 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc1.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.629 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc1.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.645 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc2.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.661 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc2.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.676 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc2.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.691 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc2.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.706 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc3.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.721 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc3.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.735 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op6, index is 0; cur kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/gradFlatten-expand/Reshape-op1, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.758 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op6, index is 0; cur kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/gradFlatten-expand/Reshape-op1, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.776 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is global_step, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.792 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is global_step, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.809 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/Mul-op0, index is 0; cur kernel is Default/Reshape-op2, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.839 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/Mul-op0, index is 0; cur kernel is Default/Reshape-op2, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.858 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc3.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.875 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc3.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.890 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/GetNext-op1, index is 1; cur kernel is Default/Reshape-op0, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.919 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/GetNext-op1, index is 1; cur kernel is Default/Reshape-op0, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.939 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1, index is 0; cur kernel is Default/Reshape-op1, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.962 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1, index is 0; cur kernel is Default/Reshape-op1, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.978 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is conv1.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.159.994 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is conv1.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, index is 0 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.160.008 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:808] CreateDeviceAddress] Status record: end create device address. graph id: 1 [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.160.201 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1129] CacheGraphOutputToFrontNodeWithIndex] Get graph backend output nodes. [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.160.238 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1137] CacheGraphOutputToFrontNodeWithIndex] Get graph front output nodes. [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:37:32.160.266 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1155] CacheGraphOutputToFrontNodeWithIndex] Backend output: Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/ReduceMean-op0 with index: 0 map to front node: Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits-op0 with index: 0 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.160.382 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:507] CompileGraph] Status record: end compile graph. graph id: 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:37:32.160.860 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:805] CompileGraphFromSegment] Compile cut segment, the cut node: @381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316:CNode_375{[0]: ValueNode Return, [1]: CNode_278} [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.161.042 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:521] Transform] Graph(kernel_graph_1) transforms actor begin, strategy:pipeline_with_execution_order [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.161.100 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2619] PersistDeviceTensorForValueNode] The device address is not exist: ValueNode_376(U) [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.161.308 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1101] BuildDataSourceActor] Create queue data source actor: kernel_graph_1_DeviceDSActor_1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.162.314 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1232] BuildLoopCountActor] Create loop count actor: kernel_graph_1_LoopCountActor [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.162.351 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1265] BuildOutputActor] Create output actor: kernel_graph_1_OutputActor [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.162.369 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1285] BuildDataPrepareActor] Create data prepare actor: kernel_graph_1_DataPrepareActor [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.162.507 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:866] CacheGraphOutputToActor] Cache graph 1 output node:Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/ReduceMean-op0 debug string:@kernel_graph_1:319{[0]: ValueNode PrimFunc_ReduceMean, [1]: 319, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), [3]: ValueNode false} with index:0 to actor:Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/ReduceMean-op0, from front node:Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits-op0 debug string:@381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316:equiv_CNode_278{[0]: ValueNode SparseSoftmaxCrossEntropyWithLogits, [1]: equiv_CNode_278, [2]: CNode_278} with index:0 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.162.532 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:809] AddSomasInfoForGraphOutput] The graph 1 output node:Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/ReduceMean-op0 with index: 0 somas enable or not: 1, somas offset: 2195456, aligned size: 512 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.162.560 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1518] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_1 start. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.162.705 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1520] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_1 end. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.163.781 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1 add input node monad device tensor store:global_step [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.163.812 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1 add input node monad device tensor store:Default/network-TrainOneStepCell/optimizer-Momentum/data-0 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.163.969 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.000 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.020 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 add input node monad device tensor store:fc3.bias [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.035 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 add input node monad device tensor store:moments.fc3.bias [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.048 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.059 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.243 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.273 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/MatMul-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.297 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op10, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.326 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 add input node monad device tensor store:fc3.weight [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.340 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 add input node monad device tensor store:moments.fc3.weight [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.351 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.362 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.415 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.438 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/BiasAdd-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.457 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 add input node monad device tensor store:fc2.bias [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.470 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 add input node monad device tensor store:moments.fc2.bias [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.481 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.492 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.543 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.579 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/MatMul-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.601 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op11, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.619 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 add input node monad device tensor store:fc2.weight [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.633 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 add input node monad device tensor store:moments.fc2.weight [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.644 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.655 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.707 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.732 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/BiasAdd-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.750 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 add input node monad device tensor store:fc1.bias [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.764 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 add input node monad device tensor store:moments.fc1.bias [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.775 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.793 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.844 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.867 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/MatMul-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.888 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op6, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.905 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 add input node monad device tensor store:fc1.weight [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.918 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 add input node monad device tensor store:moments.fc1.weight [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.929 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.939 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.164.990 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.165.013 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/Conv2D-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.165.043 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/gradConv2D-expand/Conv2DBackpropInput-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.165.062 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 add input node monad device tensor store:conv2.weight [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.165.075 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 add input node monad device tensor store:moments.conv2.weight [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.165.086 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.165.097 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.165.171 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.165.199 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv1-Conv2d/Conv2D-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.165.222 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 add input node monad device tensor store:conv1.weight [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.165.237 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 add input node monad device tensor store:moments.conv1.weight [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.165.252 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.165.266 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.165.292 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.165.321 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.165.348 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.165.372 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.165.397 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.165.420 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.165.443 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.165.462 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.166.375 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 1_actor_set_kernel_graph_1_memory_actor_insert in 24.08 us [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.166.418 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 2_actor_set_kernel_graph_1_invalid_data_arrow_elimination in 6.92999 us [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.166.675 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 3_actor_set_kernel_graph_1_multi_actor_fusion in 230.85 us [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.166.722 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 4_actor_set_kernel_graph_1_batch_data_arrow_fusion in 22.49 us [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:37:32.166.744 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:554] Transform] Graph(kernel_graph_1) transforms actor end. [WARNING] VM(164040,ffffb1c2c440,python):2024-01-10-11:37:32.168.215 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:612] CompileGraphs] [PROF]compile_func_graph costs 91578 usec. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:37:32.168.260 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:615] CompileGraphs] Status record: end compile function graph: 381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316, produce actor: kernel_graph_1 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.168.293 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end task_emit action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.168.318 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:268] SetLoopCount] Change vm_loop_flag to 0, set loop_size to 468 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.168.336 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start execute action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.168.356 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end execute action. TotalTime = 0.779781, [19] [parse]: 0.0113277 [symbol_resolve]: 0.25303, [1] [Cycle 1]: 0.252358, [1] [resolve]: 0.252332 [graph_reusing]: 7.212e-05 [meta_unpack_prepare]: 0.0012354 [pre_cconv]: 3.193e-05 [abstract_specialize]: 0.242561 [pack_expand]: 0.00046801 [auto_monad]: 0.00344734 [inline]: 3.649e-05 [pre_auto_parallel]: 5.568e-05 [pipeline_split]: 3.753e-05 [optimize]: 0.172762, [35] [py_interpret_to_execute]: 0.00113452 [rewriter_before_opt_a]: 0.00593556 [opt_a]: 0.158318, [3] [Cycle 1]: 0.125817, [30] [expand_dump_flag]: 6.302e-05 [switch_simplify]: 0.00103551 [a_1]: 0.0177029 [recompute_prepare]: 0.00015353 [updatestate_depend_eliminate]: 0.00080864 [updatestate_assign_eliminate]: 0.00025244 [updatestate_loads_eliminate]: 0.00048065 [parameter_eliminate]: 1.81e-05 [a_2]: 0.00296854 [accelerated_algorithm]: 9.57e-05 [pynative_shard]: 5.26e-05 [auto_parallel]: 5.21e-06 [parallel]: 4.468e-05 [merge_comm]: 8.285e-05 [allreduce_fusion]: 5.979e-05 [virtual_dataset]: 8.83e-05 [get_grad_eliminate_]: 7.226e-05 [virtual_output]: 6.784e-05 [merge_forward]: 0.00011838 [cell_reuse_recompute_pass]: 5.29995e-07 [cell_reuse_handle_not_recompute_node_pass]: 0.00020838 [meta_fg_expand]: 0.0336821 [after_resolve]: 0.00042344 [a_after_grad]: 0.00063539 [renormalize]: 0.0558478 [real_op_eliminate]: 0.00068402 [auto_monad_grad]: 0.00048039 [auto_monad_eliminator]: 0.00088881 [cse]: 0.00246556 [a_3]: 0.00595808 [Cycle 2]: 0.022899, [30] [expand_dump_flag]: 2.522e-05 [switch_simplify]: 0.00031889 [a_1]: 0.00903971 [recompute_prepare]: 6.091e-05 [updatestate_depend_eliminate]: 0.00055653 [updatestate_assign_eliminate]: 0.000112 [updatestate_loads_eliminate]: 0.00016981 [parameter_eliminate]: 3.65001e-06 [a_2]: 0.00099017 [accelerated_algorithm]: 7.741e-05 [pynative_shard]: 6.042e-05 [auto_parallel]: 7.29e-06 [parallel]: 9.17001e-06 [merge_comm]: 4.98e-05 [allreduce_fusion]: 3.189e-05 [virtual_dataset]: 5.469e-05 [get_grad_eliminate_]: 4.868e-05 [virtual_output]: 4.737e-05 [merge_forward]: 7.388e-05 [cell_reuse_recompute_pass]: 5.99997e-07 [cell_reuse_handle_not_recompute_node_pass]: 0.00013844 [meta_fg_expand]: 9.464e-05 [after_resolve]: 6.673e-05 [a_after_grad]: 6.313e-05 [renormalize]: 0.0088458 [real_op_eliminate]: 7.215e-05 [auto_monad_grad]: 4.63e-06 [auto_monad_eliminator]: 0.00028369 [cse]: 0.0009229 [a_3]: 0.00042616 [Cycle 3]: 0.00400743, [30] [expand_dump_flag]: 1.59e-06 [switch_simplify]: 5.216e-05 [a_1]: 0.00077892 [recompute_prepare]: 4.672e-05 [updatestate_depend_eliminate]: 0.00010835 [updatestate_assign_eliminate]: 7.148e-05 [updatestate_loads_eliminate]: 7.361e-05 [parameter_eliminate]: 2.22e-06 [a_2]: 0.00100125 [accelerated_algorithm]: 7.764e-05 [pynative_shard]: 4.77e-05 [auto_parallel]: 4.85e-06 [parallel]: 7.53e-06 [merge_comm]: 3.65e-05 [allreduce_fusion]: 2.584e-05 [virtual_dataset]: 5.529e-05 [get_grad_eliminate_]: 4.957e-05 [virtual_output]: 4.9e-05 [merge_forward]: 7.042e-05 [cell_reuse_recompute_pass]: 4.20005e-07 [cell_reuse_handle_not_recompute_node_pass]: 0.00014381 [meta_fg_expand]: 6.439e-05 [after_resolve]: 6.836e-05 [a_after_grad]: 6.426e-05 [renormalize]: 6.99947e-08 [real_op_eliminate]: 4.937e-05 [auto_monad_grad]: 2.53e-06 [auto_monad_eliminator]: 0.0001962 [cse]: 0.00026985 [a_3]: 0.0004158 [py_interpret_to_execute_after_opt_a]: 9.238e-05 [slice_cell_reuse_recomputed_activation]: 2.19e-06 [rewriter_after_opt_a]: 0.00197501 [convert_after_rewriter]: 9.142e-05 [order_py_execute_after_rewriter]: 6.095e-05 [opt_b]: 0.00197378, [1] [Cycle 1]: 0.00196713, [7] [b_1]: 0.00133941 [b_2]: 5.79e-05 [updatestate_depend_eliminate]: 7.217e-05 [updatestate_assign_eliminate]: 6.879e-05 [updatestate_loads_eliminate]: 7.387e-05 [renormalize]: 4.313e-05 [cse]: 0.00026158 [cconv]: 6.18e-05 [opt_after_cconv]: 0.0007487, [1] [Cycle 1]: 0.00074219, [7] [c_1]: 0.00018709 [parameter_eliminate]: 2.05e-06 [updatestate_depend_eliminate]: 8.392e-05 [updatestate_assign_eliminate]: 7.092e-05 [updatestate_loads_eliminate]: 7.165e-05 [cse]: 0.0002432 [renormalize]: 4.031e-05 [remove_dup_value]: 0.00029383 [tuple_transform]: 0.00054187, [1] [Cycle 1]: 0.00053536, [3] [d_1]: 0.00032429 [d_2]: 0.00014935 [renormalize]: 3.93e-05 [add_cache_embedding]: 0.00014665 [add_recomputation]: 0.0006578 [cse_after_recomputation]: 0.00022874, [1] [Cycle 1]: 0.0002221, [1] [cse]: 0.00021222 [environ_conv]: 8.705e-05 [label_micro_interleaved_index]: 2.55e-06 [label_fine_grained_interleaved_index]: 1.51e-06 [assign_add_opt]: 3.979e-05 [slice_recompute_activation]: 2.03e-06 [micro_interleaved_order_control]: 1.31e-06 [full_micro_interleaved_order_control]: 1.46e-06 [comp_comm_scheduling]: 1.33e-06 [reorder_send_recv_between_fp_bp]: 1.79e-06 [comm_op_add_attrs]: 8.79998e-07 [add_comm_op_reuse_tag]: 2.12e-05 [overlap_opt_shard_in_pipeline]: 1.747e-05 [grouped_pairwise_exchange_alltoall]: 5.515e-05 [overlap_recompute_and_grad_model_parallel]: 1.42e-06 [overlap_grad_matmul_and_grad_allreduce]: 1.611e-05 [split_matmul_comm_elemetwise]: 1.369e-05 [split_layernorm_comm]: 1.48e-06 [process_send_recv_for_ge]: 2.48e-06 [handle_group_info]: 5.4e-07 [auto_monad_reorder]: 0.00038251 [get_jit_bprop_graph]: 2.862e-05 [eliminate_special_op_node]: 0.00091529 [validate]: 0.0003145 [distribtued_split]: 0.00048861 [task_emit]: 0.0922295 [execute]: 3.092e-05 Sums parse : 0.011328s : 1.47% symbol_resolve.resolve : 0.252332s : 32.69% graph_reusing : 0.000072s : 0.01% meta_unpack_prepare : 0.001235s : 0.16% pre_cconv : 0.000032s : 0.00% abstract_specialize : 0.242561s : 31.42% pack_expand : 0.000468s : 0.06% auto_monad : 0.003447s : 0.45% inline : 0.000036s : 0.00% pre_auto_parallel : 0.000056s : 0.01% pipeline_split : 0.000038s : 0.00% optimize.py_interpret_to_execute : 0.001135s : 0.15% optimize.rewriter_before_opt_a : 0.005936s : 0.77% optimize.opt_a.expand_dump_flag : 0.000090s : 0.01% optimize.opt_a.switch_simplify : 0.001407s : 0.18% optimize.opt_a.a_1 : 0.027522s : 3.56% optimize.opt_a.recompute_prepare : 0.000261s : 0.03% optimize.opt_a.updatestate_depend_eliminate : 0.001474s : 0.19% optimize.opt_a.updatestate_assign_eliminate : 0.000436s : 0.06% optimize.opt_a.updatestate_loads_eliminate : 0.000724s : 0.09% optimize.opt_a.parameter_eliminate : 0.000024s : 0.00% optimize.opt_a.a_2 : 0.004960s : 0.64% optimize.opt_a.accelerated_algorithm : 0.000251s : 0.03% optimize.opt_a.pynative_shard : 0.000161s : 0.02% optimize.opt_a.auto_parallel : 0.000017s : 0.00% optimize.opt_a.parallel : 0.000061s : 0.01% optimize.opt_a.merge_comm : 0.000169s : 0.02% optimize.opt_a.allreduce_fusion : 0.000118s : 0.02% optimize.opt_a.virtual_dataset : 0.000198s : 0.03% optimize.opt_a.get_grad_eliminate_ : 0.000171s : 0.02% optimize.opt_a.virtual_output : 0.000164s : 0.02% optimize.opt_a.merge_forward : 0.000263s : 0.03% optimize.opt_a.cell_reuse_recompute_pass : 0.000002s : 0.00% optimize.opt_a.cell_reuse_handle_not_recompute_node_pass : 0.000491s : 0.06% optimize.opt_a.meta_fg_expand : 0.033841s : 4.38% optimize.opt_a.after_resolve : 0.000559s : 0.07% optimize.opt_a.a_after_grad : 0.000763s : 0.10% optimize.opt_a.renormalize : 0.064694s : 8.38% optimize.opt_a.real_op_eliminate : 0.000806s : 0.10% optimize.opt_a.auto_monad_grad : 0.000488s : 0.06% optimize.opt_a.auto_monad_eliminator : 0.001369s : 0.18% optimize.opt_a.cse : 0.003658s : 0.47% optimize.opt_a.a_3 : 0.006800s : 0.88% optimize.py_interpret_to_execute_after_opt_a : 0.000092s : 0.01% optimize.slice_cell_reuse_recomputed_activation : 0.000002s : 0.00% optimize.rewriter_after_opt_a : 0.001975s : 0.26% optimize.convert_after_rewriter : 0.000091s : 0.01% optimize.order_py_execute_after_rewriter : 0.000061s : 0.01% optimize.opt_b.b_1 : 0.001339s : 0.17% optimize.opt_b.b_2 : 0.000058s : 0.01% optimize.opt_b.updatestate_depend_eliminate : 0.000072s : 0.01% optimize.opt_b.updatestate_assign_eliminate : 0.000069s : 0.01% optimize.opt_b.updatestate_loads_eliminate : 0.000074s : 0.01% optimize.opt_b.renormalize : 0.000043s : 0.01% optimize.opt_b.cse : 0.000262s : 0.03% optimize.cconv : 0.000062s : 0.01% optimize.opt_after_cconv.c_1 : 0.000187s : 0.02% optimize.opt_after_cconv.parameter_eliminate : 0.000002s : 0.00% optimize.opt_after_cconv.updatestate_depend_eliminate : 0.000084s : 0.01% optimize.opt_after_cconv.updatestate_assign_eliminate : 0.000071s : 0.01% optimize.opt_after_cconv.updatestate_loads_eliminate : 0.000072s : 0.01% optimize.opt_after_cconv.cse : 0.000243s : 0.03% optimize.opt_after_cconv.renormalize : 0.000040s : 0.01% optimize.remove_dup_value : 0.000294s : 0.04% optimize.tuple_transform.d_1 : 0.000324s : 0.04% optimize.tuple_transform.d_2 : 0.000149s : 0.02% optimize.tuple_transform.renormalize : 0.000039s : 0.01% optimize.add_cache_embedding : 0.000147s : 0.02% optimize.add_recomputation : 0.000658s : 0.09% optimize.cse_after_recomputation.cse : 0.000212s : 0.03% optimize.environ_conv : 0.000087s : 0.01% optimize.label_micro_interleaved_index : 0.000003s : 0.00% optimize.label_fine_grained_interleaved_index : 0.000002s : 0.00% optimize.assign_add_opt : 0.000040s : 0.01% optimize.slice_recompute_activation : 0.000002s : 0.00% optimize.micro_interleaved_order_control : 0.000001s : 0.00% optimize.full_micro_interleaved_order_control : 0.000001s : 0.00% optimize.comp_comm_scheduling : 0.000001s : 0.00% optimize.reorder_send_recv_between_fp_bp : 0.000002s : 0.00% optimize.comm_op_add_attrs : 0.000001s : 0.00% optimize.add_comm_op_reuse_tag : 0.000021s : 0.00% optimize.overlap_opt_shard_in_pipeline : 0.000017s : 0.00% optimize.grouped_pairwise_exchange_alltoall : 0.000055s : 0.01% optimize.overlap_recompute_and_grad_model_parallel : 0.000001s : 0.00% optimize.overlap_grad_matmul_and_grad_allreduce : 0.000016s : 0.00% optimize.split_matmul_comm_elemetwise : 0.000014s : 0.00% optimize.split_layernorm_comm : 0.000001s : 0.00% optimize.process_send_recv_for_ge : 0.000002s : 0.00% optimize.handle_group_info : 0.000001s : 0.00% auto_monad_reorder : 0.000383s : 0.05% get_jit_bprop_graph : 0.000029s : 0.00% eliminate_special_op_node : 0.000915s : 0.12% validate : 0.000314s : 0.04% distribtued_split : 0.000489s : 0.06% task_emit : 0.092230s : 11.95% execute : 0.000031s : 0.00% Time group info: ------[substitution.] 0.261617 4489 0.09% : 0.000227s : 57: substitution.arithmetic_simplify 0.02% : 0.000042s : 16: substitution.cast_eliminate 0.02% : 0.000048s : 55: substitution.depend_value_elim 0.01% : 0.000017s : 16: substitution.environ_get_add_eliminate 0.00% : 0.000007s : 8: substitution.environ_get_depend_swap 0.01% : 0.000022s : 32: substitution.environ_get_eliminate 0.02% : 0.000042s : 16: substitution.environ_get_set_eliminate 0.01% : 0.000037s : 94: substitution.float_depend_g_call 0.00% : 0.000008s : 16: substitution.float_environ_get_switch 0.01% : 0.000013s : 14: substitution.float_tuple_getitem_switch 94.77% : 0.247926s : 233: substitution.getattr_setattr_resolve 0.02% : 0.000054s : 120: substitution.graph_param_transform 0.00% : 0.000007s : 20: substitution.incorporate_call 0.00% : 0.000005s : 20: substitution.incorporate_call_switch 3.96% : 0.010365s : 377: substitution.inline 0.01% : 0.000014s : 34: substitution.less_batch_normalization 0.01% : 0.000031s : 160: substitution.load_eliminater 0.15% : 0.000388s : 505: substitution.meta_unpack_prepare 0.02% : 0.000055s : 72: substitution.minmaximum_grad 0.01% : 0.000015s : 8: substitution.partial_defer_inline 0.05% : 0.000124s : 94: substitution.partial_eliminate 0.00% : 0.000009s : 120: substitution.partial_unused_args_eliminate 0.02% : 0.000046s : 31: substitution.real_op_eliminate 0.01% : 0.000014s : 32: substitution.reduce_all_const_elim 0.01% : 0.000035s : 338: substitution.remove_not_recompute_node 0.13% : 0.000336s : 264: substitution.replace_applicator 0.01% : 0.000034s : 142: substitution.replace_old_param 0.00% : 0.000009s : 2: substitution.reshape_eliminate 0.00% : 0.000006s : 10: substitution.set_cell_output_no_recompute 0.00% : 0.000005s : 2: substitution.specialize_transform 0.01% : 0.000022s : 32: substitution.split_environ_get_set_with_tuple_value 0.01% : 0.000026s : 31: substitution.switch_simplify 0.07% : 0.000187s : 76: substitution.tuple_list_convert_item_index_to_positive 0.04% : 0.000093s : 92: substitution.tuple_list_get_item_const_eliminator 0.06% : 0.000164s : 92: substitution.tuple_list_get_item_depend_reorder 0.19% : 0.000493s : 283: substitution.tuple_list_get_item_eliminator 0.05% : 0.000118s : 92: substitution.tuple_list_get_set_item_eliminator 0.10% : 0.000272s : 416: substitution.updatestate_pure_node_eliminater 0.12% : 0.000302s : 467: substitution.updatestate_useless_node_eliminater ------[renormalize.] 0.064673 4 58.50% : 0.037832s : 2: renormalize.infer 41.50% : 0.026841s : 2: renormalize.specialize ------[replace.] 0.008157 886 0.07% : 0.000006s : 1: replace.arithmetic_simplify 1.01% : 0.000083s : 16: replace.cast_eliminate 0.61% : 0.000050s : 10: replace.depend_value_elim 1.50% : 0.000122s : 8: replace.environ_get_set_eliminate 36.43% : 0.002972s : 210: replace.getattr_setattr_resolve 30.09% : 0.002455s : 341: replace.inline 0.39% : 0.000032s : 1: replace.meta_unpack_prepare 4.82% : 0.000393s : 32: replace.partial_eliminate 1.87% : 0.000153s : 31: replace.real_op_eliminate 1.90% : 0.000155s : 9: replace.replace_applicator 3.41% : 0.000278s : 31: replace.switch_simplify 1.31% : 0.000107s : 16: replace.tuple_list_get_item_depend_reorder 16.33% : 0.001332s : 179: replace.tuple_list_get_item_eliminator 0.25% : 0.000020s : 1: replace.updatestate_useless_node_eliminater ------[match.] 0.258300 886 0.00% : 0.000012s : 1: match.arithmetic_simplify 0.02% : 0.000042s : 16: match.cast_eliminate 0.00% : 0.000008s : 10: match.depend_value_elim 0.01% : 0.000036s : 8: match.environ_get_set_eliminate 95.76% : 0.247337s : 210: match.getattr_setattr_resolve 3.93% : 0.010151s : 341: match.inline 0.05% : 0.000126s : 1: match.meta_unpack_prepare 0.04% : 0.000096s : 32: match.partial_eliminate 0.02% : 0.000046s : 31: match.real_op_eliminate 0.01% : 0.000033s : 9: match.replace_applicator 0.01% : 0.000026s : 31: match.switch_simplify 0.02% : 0.000059s : 16: match.tuple_list_get_item_depend_reorder 0.12% : 0.000321s : 179: match.tuple_list_get_item_eliminator 0.00% : 0.000006s : 1: match.updatestate_useless_node_eliminater ------[func_graph_cloner_run.] 0.042240 648 71.50% : 0.030203s : 267: func_graph_cloner_run.FuncGraphClonerGraph 28.50% : 0.012038s : 381: func_graph_cloner_run.FuncGraphSpecializer ------[meta_graph.] 0.000000 0 ------[manager.] 0.000000 0 ------[pynative] 0.000000 0 ------[others.] 0.301081 170 4.04% : 0.012160s : 103: opt.transform.opt_a 0.44% : 0.001323s : 23: opt.transform.opt_b 84.19% : 0.253495s : 6: opt.transform.opt_resolve 0.38% : 0.001153s : 1: opt.transforms.meta_unpack_prepare 10.65% : 0.032055s : 30: opt.transforms.opt_a 0.06% : 0.000184s : 1: opt.transforms.opt_after_cconv 0.02% : 0.000056s : 1: opt.transforms.opt_b 0.16% : 0.000470s : 2: opt.transforms.opt_trans_graph 0.06% : 0.000184s : 3: opt.transforms.special_op_eliminate [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.169.091 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1385] Run] End [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.169.124 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:846] SaveCompiledGraph] Save compiled func graph(381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316) phase(train.1704857851222130944.281471961594384.0)! [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.169.146 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:864] SaveCompiledGraph] End save compiled func graph! [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.169.160 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:942] CleanCompileRes] Clean compile resource start [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.179.312 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:956] CleanCompileRes] Clean compile resource end [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.179.346 [mindspore/ccsrc/pipeline/jit/ps/event_message_print.cc:37] PrintEventMessage] End compiling '_DataWrapper.construct'. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.179.361 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1039] CompileInner] Finish compiling. [WARNING] ME(164040:281473664074816,MainProcess):2024-01-10-11:37:32.180.447 [mindspore/parallel/_utils.py:259] You are suggested to use mindspore.context.set_auto_parallel_context(parameter_broadcast=True) or mindspore.common.set_seed() to share parameters among multi-devices. [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:37:32.183.021 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:37:32.183.100 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:37:32.183.138 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_1 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.183.589 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[1], dtype=Int32, value=[1]), output index: 0 device address:0x32f8fcb0 [INFO] PRE_ACT(164040,fffe4b7460f0,python):2024-01-10-11:37:32.183.660 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:237] SetMemAllocUintSize] Set mem alloc unit size, common 1073741824 persistent 1073741824 [INFO] DEVICE(164040,fffe4b7460f0,python):2024-01-10-11:37:32.183.681 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_pool.cc:124] AllocDeviceMem] Malloc Memory for Pool, size: 1073741824 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.183.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode (32, 16, 5, 5) [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.184.008 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode (16, 6, 5, 5) [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.184.100 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Float32, value=0.25), output index: 0 device address:0x33028360 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.184.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[32, 1], dtype=Float32, value=[...]), output index: 0 device address:0x33188840 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.184.260 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), output index: 0 device address:0x31a800a0 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.184.417 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode 1 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.184.501 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode (32, 6, 14, 14) [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.184.587 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode false [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.184.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10]), output index: 0 device address:0x33093c50 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.184.744 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode (6, 1, 5, 5) [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.184.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10]), output index: 0 device address:0x32e824a0 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.184.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Int64, value=10), output index: 0 device address:0x31a72a40 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.184.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode -1 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.185.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Float32, value=0), output index: 0 device address:0x31668330 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.185.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode true [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.185.195 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Float32, value=1), output index: 0 device address:0x31aaf500 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.185.265 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode 0 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.185.343 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[1], dtype=Int64, value=[32]), output index: 0 device address:0x31a930a0 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.185.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc3.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.185.535 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc3.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.185.630 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc2.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.185.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc2.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.185.908 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc1.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.186.001 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc1.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.186.128 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_conv2.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.186.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_conv1.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.186.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_global_step, device type:2 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.186.412 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc3.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.186.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_learning_rate, device type:2 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.186.590 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_momentum, device type:2 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.186.678 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc3.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.186.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc2.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.186.858 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc2.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.186.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc1.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.187.044 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc1.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.187.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.conv2.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:37:32.187.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.conv1.weight, device type:2 [INFO] PRE_ACT(164040,fffe4b7460f0,python):2024-01-10-11:37:32.187.380 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:237] SetMemAllocUintSize] Set mem alloc unit size, common 1073741824 persistent 1073741824 [INFO] DEVICE(164040,fffe4b7460f0,python):2024-01-10-11:37:32.187.404 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_pool.cc:124] AllocDeviceMem] Malloc Memory for Pool, size: 1073741824 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:37:32.187.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] GE(164040,python):2024-01-10-11:37:32.269.224 [graph_var_manager.cc:1424][EVENT]167130 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:32.269.324 [graph_manager.cc:1248][EVENT]167130 PreRun:PreRun start: graph node size 2, session id 2, graph id 1, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:32.270.296 [atrace_api.c:28](tid:167130) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:32.270.372 [trace_rb_log.c:84](tid:167130) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:32.270.386 [atrace_api.c:32](tid:167130) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:32.270.405 [client_manager.cpp:157][SetProfilingCallback][tid:167130] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:32.271.307 [parallel_partitioner.cc:165][EVENT]167130 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.271.346 [parallel_partitioner.cc:178][EVENT]167130 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.271.392 [graph_prepare.cc:1378][EVENT]167130 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.272.111 [graph_manager.cc:1050][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [735] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.272.142 [graph_manager.cc:1052][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.272.214 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.272.242 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.272.340 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [87] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.272.355 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.272.407 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.272.421 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.272.431 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.272.536 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.272.581 [graph_manager.cc:1054][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [426] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.272.803 [graph_manager.cc:1055][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [209] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.273.711 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.273.742 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.273.754 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.273.763 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [255] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.273.772 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.273.785 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.273.797 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [48] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.273.806 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.273.814 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.275.201 [graph_manager.cc:1056][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2375] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.275.264 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.275.283 [graph_prepare.cc:1982][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [46] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.275.627 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.275.652 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.275.663 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.275.676 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [130] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.275.685 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.275.698 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.275.706 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.275.715 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [6] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.275.732 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.275.771 [graph_prepare.cc:1983][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [475] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.275.795 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.275.806 [graph_prepare.cc:1984][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.275.820 [graph_prepare.cc:1985][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.275.838 [graph_prepare.cc:1986][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.275.850 [graph_prepare.cc:1987][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.275.865 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.275.876 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.275.890 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.275.963 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.275.978 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.275.988 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.275.996 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.276.005 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.276.014 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.276.022 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.276.030 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.276.039 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.276.047 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.276.055 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.276.063 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.276.071 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.276.087 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.276.096 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.276.104 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.276.126 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.276.138 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.276.167 [graph_prepare.cc:1988][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [307] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.276.179 [graph_manager.cc:1065][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [943] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.288.037 [graph_manager.cc:1077][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11840] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.288.105 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.288.157 [graph_manager.cc:1080][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [82] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.290.872 [graph_manager.cc:1081][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [2700] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.290.915 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.290.929 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.290.941 [graph_manager.cc:1082][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [33] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.290.969 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.290.982 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.290.995 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.026 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.040 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.052 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.065 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.111 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [26] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.130 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.166 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [26] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.190 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.206 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.218 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.228 [graph_manager.cc:2700][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [263] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.309 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [0] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.291.322 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.291.331 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.291.340 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.291.348 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.291.357 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CastRemovePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.291.365 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.291.374 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.291.382 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.291.390 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.291.398 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [6] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.291.407 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.291.415 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [5] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.291.423 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.291.431 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.291.441 [graph_manager.cc:2741][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [195] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.456 [graph_manager.cc:2752][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.479 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.490 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.506 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.521 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.533 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.544 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.562 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.575 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.589 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.599 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.615 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.626 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.644 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.656 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.665 [graph_manager.cc:2810][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [189] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.688 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.291.699 [graph_manager.cc:2821][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [25] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.726 [graph_manager.cc:1087][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [769] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.858 [graph_manager.cc:1088][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [119] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.894 [graph_manager.cc:1089][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.913 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.291.951 [graph_manager.cc:1097][EVENT]167130 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:32.291.973 [graph_manager.cc:3325][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.292.088 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.292.103 [engine_place.cc:144][EVENT]167130 Run:The time cost of aicpu_ascend_kernel::CheckSupported is [42] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.292.168 [graph_manager.cc:3351][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [183] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.292.188 [graph_manager.cc:3364][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.292.245 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.292.260 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.292.380 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [110] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.292.406 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.292.444 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.292.476 [graph_manager.cc:3405][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [275] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.292.493 [graph_manager.cc:3412][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.307 [graph_manager.cc:3422][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [1799] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.343 [graph_manager.cc:3428][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.457 [graph_manager.cc:3467][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [95] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.475 [graph_manager.cc:3377][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [2275] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.490 [graph_manager.cc:1106][EVENT]167130 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [2523] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.503 [graph_manager.cc:1115][EVENT]167130 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:32.294.525 [graph_manager.cc:1130][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.556 [graph_manager.cc:1131][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.588 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.603 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.613 [graph_manager.cc:2837][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.698 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.294.712 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [0] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.294.722 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.294.731 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.294.739 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [30] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.294.748 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:37:32.294.758 [graph_manager.cc:2864][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [128] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.769 [graph_manager.cc:2872][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.787 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.802 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.816 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.830 [compile_nodes_pass.cc:88][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.839 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.850 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.928 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [69] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.957 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.971 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.294.984 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.004 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.014 [graph_manager.cc:2927][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [230] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.026 [graph_manager.cc:2937][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.053 [graph_manager.cc:2943][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.067 [graph_manager.cc:2950][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.230 [graph_manager.cc:2958][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.260 [graph_manager.cc:1132][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [681] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.371 [graph_manager.cc:1135][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [97] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.407 [graph_manager.cc:2975][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.513 [graph_manager.cc:2981][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [92] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.533 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.544 [graph_manager.cc:2986][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.554 [graph_manager.cc:1136][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [167] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.647 [graph_manager.cc:3555][EVENT]167130 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [67] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.699 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.713 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.803 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [80] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.826 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.857 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.876 [graph_builder.cc:865][EVENT]167130 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [201] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.295.972 [graph_builder.cc:288][EVENT]167130 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [78] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.296.170 [graph_builder.cc:293][EVENT]167130 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [175] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.296.350 [model_builder.cc:1133][EVENT]167130 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [91] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.296.610 [block_mem_assigner.cc:4069][EVENT]167507 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164040,python):2024-01-10-11:37:32.296.606 [block_mem_assigner.cc:4069][EVENT]167506 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164040,python):2024-01-10-11:37:32.297.108 [graph_mem_assigner.cc:2166][EVENT]167130 SetInputOffset:[IMAS]AfterAssignMemory : online_1 memoffset[132096], memtype[2], theory_min[264192], zero_copy[132096], total_size[132096], no_reuse[132096], streams[1], topo_mode[DFS], mop[], io_reuse[0:0], alloc_mode[] [INFO] GE(164040,python):2024-01-10-11:37:32.297.196 [model_builder.cc:1144][EVENT]167130 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [824] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.297.220 [model_builder.cc:1152][EVENT]167130 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.297.235 [model_builder.cc:1157][EVENT]167130 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.297.345 [model_builder.cc:1167][EVENT]167130 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [96] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.297.363 [model_builder.cc:1174][EVENT]167130 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.297.383 [model_builder.cc:1180][EVENT]167130 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.297.417 [model_builder.cc:1184][EVENT]167130 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.297.435 [graph_builder.cc:304][EVENT]167130 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [1242] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:32.297.672 [logger.cc:1071] 167130 ModelBindStream: model_id=64, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:37:32.297.753 [task_generator.cc:804][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.297.812 [task_generator.cc:805][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [43] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.298.355 [task_generator.cc:814][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [529] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.298.370 [task_generator.cc:954][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [622] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.298.424 [task_generator.cc:967][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [32] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:32.298.442 [logger.cc:1084] 167130 ModelUnbindStream: model_id=64, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:37:32.298.498 [graph_builder.cc:310][EVENT]167130 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [1051] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.298.607 [graph_manager.cc:1152][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3033] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.298.624 [graph_manager.cc:1164][EVENT]167130 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:32.298.656 [graph_manager.cc:1271][EVENT]167130 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [27434] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.298.667 [graph_manager.cc:1272][EVENT]167130 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:37:32.298.978 [atrace_api.c:93](tid:167130) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:37:32.298.997 [atrace_api.c:95](tid:167130) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:37:32.299.726 [model_introduction.cc:236][EVENT]167130 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(164040,python):2024-01-10-11:37:32.299.750 [model_introduction.cc:294][EVENT]167130 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(164040,python):2024-01-10-11:37:32.299.764 [model_introduction.cc:366][EVENT]167130 Data:model io_info size:114 [INFO] GE(164040,python):2024-01-10-11:37:32.303.387 [graph_converter.cc:838][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1369] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.303.615 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [180] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.304.046 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [404] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.304.135 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [60] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.304.152 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [78] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.304.194 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.304.224 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.304.253 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.304.324 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [59] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.304.385 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [49] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.304.396 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [60] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.304.426 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.304.450 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.304.463 [graph_converter.cc:849][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1033] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.304.662 [graph_converter.cc:853][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [190] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.305.277 [graph_converter.cc:857][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [599] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.305.405 [graph_converter.cc:862][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [91] micro second. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:37:32.310.698 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 101 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] GE(164040,python):2024-01-10-11:37:32.395.331 [graph_var_manager.cc:1424][EVENT]167129 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:32.395.435 [graph_manager.cc:1248][EVENT]167129 PreRun:PreRun start: graph node size 6, session id 3, graph id 2, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:32.395.768 [atrace_api.c:28](tid:167129) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:32.395.800 [trace_rb_log.c:84](tid:167129) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:32.395.813 [atrace_api.c:32](tid:167129) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:32.395.830 [client_manager.cpp:157][SetProfilingCallback][tid:167129] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:32.396.225 [parallel_partitioner.cc:165][EVENT]167129 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.396.265 [parallel_partitioner.cc:178][EVENT]167129 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.396.312 [graph_prepare.cc:1378][EVENT]167129 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.396.481 [graph_manager.cc:1050][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [187] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.396.505 [graph_manager.cc:1052][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.396.677 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.396.709 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.396.758 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.396.771 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.396.816 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.396.830 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.396.849 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.396.953 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.396.973 [graph_manager.cc:1054][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [455] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.397.220 [graph_manager.cc:1055][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [208] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.398.756 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:37:32.398.788 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.398.799 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [5] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.398.809 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [461] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.398.818 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.398.827 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:37:32.398.836 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [64] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.398.844 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [22] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.398.853 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [9] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.401.318 [graph_manager.cc:1056][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [4077] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.401.392 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.401.411 [graph_prepare.cc:1982][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [57] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.402.000 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:37:32.402.030 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.041 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.051 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [363] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.059 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.068 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [4] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:37:32.402.076 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [9] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.085 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.093 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.119 [graph_prepare.cc:1983][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [695] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.402.154 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.402.166 [graph_prepare.cc:1984][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.402.180 [graph_prepare.cc:1985][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.402.194 [graph_prepare.cc:1986][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.402.205 [graph_prepare.cc:1987][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.402.219 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.402.231 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.402.244 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.402.357 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.369 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondPass is [6] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.377 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.386 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [3] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.394 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.403 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.411 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.419 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.427 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of StopGradientPass is [3] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.436 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.444 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.452 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.460 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.468 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.482 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.491 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.402.514 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.402.527 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.402.563 [graph_prepare.cc:1988][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [349] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.402.576 [graph_manager.cc:1065][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1223] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.415.458 [graph_manager.cc:1077][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12862] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.415.530 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.415.582 [graph_manager.cc:1080][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [85] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.423.394 [graph_manager.cc:1081][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [7792] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.423.438 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.423.454 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.423.465 [graph_manager.cc:1082][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.423.498 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.423.514 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.423.529 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.423.563 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [24] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.423.577 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.423.593 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.423.606 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.423.651 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.423.672 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.423.713 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.423.745 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.423.762 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.423.774 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.423.783 [graph_manager.cc:2700][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [291] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.423.931 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.423.945 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.423.954 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.423.963 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.423.971 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.423.980 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.423.988 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.423.996 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.424.005 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.424.013 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [4] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.424.021 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.424.029 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.424.037 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.424.046 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.424.054 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [4] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.424.063 [graph_manager.cc:2741][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [260] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.072 [graph_manager.cc:2752][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.112 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.125 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.144 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.158 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.169 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.181 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.201 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.216 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.229 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.239 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.252 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.263 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.284 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.296 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.305 [graph_manager.cc:2810][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [197] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.338 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.424.349 [graph_manager.cc:2821][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.377 [graph_manager.cc:1087][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [892] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.512 [graph_manager.cc:1088][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [121] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.556 [graph_manager.cc:1089][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.575 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.424.590 [graph_manager.cc:1097][EVENT]167129 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:32.424.612 [graph_manager.cc:3325][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.425.800 [engine_place.cc:144][EVENT]167129 Run:The time cost of AIcoreEngine::CheckSupported is [1036] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.425.831 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.425.841 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.425.987 [graph_manager.cc:3351][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [1353] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.426.007 [graph_manager.cc:3364][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.426.075 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.426.093 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.426.296 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [193] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.426.345 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.426.395 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.426.430 [graph_manager.cc:3405][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [410] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.426.448 [graph_manager.cc:3412][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.437.323 [graph_manager.cc:3422][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [10861] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.437.362 [graph_manager.cc:3428][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.437.506 [graph_manager.cc:3467][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [123] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.437.524 [graph_manager.cc:3377][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [11505] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.437.539 [graph_manager.cc:1106][EVENT]167129 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [12935] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.437.553 [graph_manager.cc:1115][EVENT]167129 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:32.437.576 [graph_manager.cc:1130][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.437.607 [graph_manager.cc:1131][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.437.641 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.437.660 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.437.670 [graph_manager.cc:2837][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.437.784 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.437.798 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.437.808 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.437.816 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.437.825 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.437.834 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [6] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.437.843 [graph_manager.cc:2864][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [130] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.437.855 [graph_manager.cc:2872][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.437.875 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.437.889 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.437.905 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.437.919 [compile_nodes_pass.cc:88][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.437.929 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.437.939 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.034 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [86] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.063 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.076 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.090 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.110 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.120 [graph_manager.cc:2927][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [248] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.133 [graph_manager.cc:2937][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.147 [graph_manager.cc:2943][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.158 [graph_manager.cc:2950][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.332 [graph_manager.cc:2958][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [46] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.366 [graph_manager.cc:1132][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [735] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.436 [graph_manager.cc:1135][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [57] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.470 [graph_manager.cc:2975][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.501 [graph_manager.cc:2981][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.515 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.526 [graph_manager.cc:2986][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.535 [graph_manager.cc:1136][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [82] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.665 [graph_manager.cc:3555][EVENT]167129 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [95] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.762 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.780 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.943 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [153] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.438.982 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.439.023 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [29] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.439.047 [graph_builder.cc:865][EVENT]167129 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [325] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:32.439.393 [logger.cc:1071] 167129 ModelBindStream: model_id=832, stream_id=1089, flag=0. [INFO] GE(164040,python):2024-01-10-11:37:32.439.436 [task_generator.cc:804][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [105] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.439.505 [task_generator.cc:805][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [57] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.440.330 [task_generator.cc:814][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [810] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.440.345 [task_generator.cc:954][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1014] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.440.409 [task_generator.cc:967][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [37] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:32.440.428 [logger.cc:1084] 167129 ModelUnbindStream: model_id=832, stream_id=1089, [INFO] GE(164040,python):2024-01-10-11:37:32.440.622 [graph_manager.cc:1152][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2060] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.440.642 [graph_manager.cc:1164][EVENT]167129 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:32.440.674 [graph_manager.cc:1271][EVENT]167129 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [44542] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.440.686 [graph_manager.cc:1272][EVENT]167129 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:37:32.440.995 [atrace_api.c:93](tid:167129) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:37:32.441.009 [atrace_api.c:95](tid:167129) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:37:32.447.815 [graph_converter.cc:838][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [2079] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.447.984 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [123] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.448.620 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [611] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.448.916 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [269] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.448.940 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [295] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.449.218 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [265] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.449.266 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.449.302 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.449.552 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [236] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.449.653 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [81] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.449.669 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [97] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.449.713 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [34] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.449.744 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.449.787 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.449.894 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [96] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.449.981 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [73] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.449.992 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [85] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.450.023 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.450.051 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.450.064 [graph_converter.cc:849][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [2210] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.450.363 [graph_converter.cc:853][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [288] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.451.326 [graph_converter.cc:857][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [947] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.451.525 [graph_converter.cc:862][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [170] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.526.020 [graph_var_manager.cc:1424][EVENT]167130 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:32.526.122 [graph_manager.cc:1248][EVENT]167130 PreRun:PreRun start: graph node size 4, session id 4, graph id 3, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:32.526.436 [atrace_api.c:28](tid:167130) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:32.526.470 [trace_rb_log.c:84](tid:167130) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:32.526.483 [atrace_api.c:32](tid:167130) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:32.526.500 [client_manager.cpp:157][SetProfilingCallback][tid:167130] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:32.526.871 [parallel_partitioner.cc:165][EVENT]167130 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.526.909 [parallel_partitioner.cc:178][EVENT]167130 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.526.954 [graph_prepare.cc:1378][EVENT]167130 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.527.112 [graph_manager.cc:1050][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [173] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.527.135 [graph_manager.cc:1052][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.527.271 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.527.300 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.527.372 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.527.386 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.527.432 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.527.446 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.527.463 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.527.564 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.527.584 [graph_manager.cc:1054][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [436] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.527.803 [graph_manager.cc:1055][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [206] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.528.980 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:32.529.011 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.529.023 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.529.033 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [445] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.529.042 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.529.051 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:32.529.060 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [9] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.529.068 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.529.077 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.532.758 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:32.532.792 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.532.804 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.532.813 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [373] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.532.822 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.532.831 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:32.532.850 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.532.860 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.532.868 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.534.193 [graph_manager.cc:1056][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [6369] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.534.262 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.534.280 [graph_prepare.cc:1982][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [50] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.534.827 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:32.534.855 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.534.866 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.534.875 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [300] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.534.884 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.534.892 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:32.534.901 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.534.909 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.534.918 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.534.967 [graph_prepare.cc:1983][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [674] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.534.995 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.535.006 [graph_prepare.cc:1984][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.535.020 [graph_prepare.cc:1985][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.535.034 [graph_prepare.cc:1986][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.535.046 [graph_prepare.cc:1987][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.535.060 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.535.083 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.535.097 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.535.190 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.535.202 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.535.211 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.535.220 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.535.228 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.535.237 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.535.245 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.535.253 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.535.262 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.535.270 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.535.278 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.535.287 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.535.295 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.535.303 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.535.311 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.535.319 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:32.535.341 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.535.355 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.535.388 [graph_prepare.cc:1988][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [334] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.535.401 [graph_manager.cc:1065][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1173] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.547.567 [graph_manager.cc:1077][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12144] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.547.679 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.547.733 [graph_manager.cc:1080][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [116] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.558.230 [graph_manager.cc:1081][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [10480] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.558.276 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.558.294 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.558.306 [graph_manager.cc:1082][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.558.338 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.558.353 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.558.370 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.558.551 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [166] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.558.569 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.558.675 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [96] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.558.691 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.558.742 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [41] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.558.767 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.558.787 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.558.915 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [117] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.558.935 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.558.949 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.558.959 [graph_manager.cc:2700][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [627] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.559.278 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [6] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.559.308 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.559.318 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [6] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.559.328 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.559.336 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.559.345 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CastRemovePass is [46] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.559.354 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [5] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.559.362 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [5] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.559.370 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [13] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.559.378 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.559.387 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [19] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.559.395 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [67] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.559.403 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [22] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.559.411 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [9] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.559.420 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.559.429 [graph_manager.cc:2741][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [452] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.559.438 [graph_manager.cc:2752][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.559.463 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.559.476 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.559.499 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.559.514 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.559.527 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.559.541 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.559.565 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.559.590 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.559.603 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.559.613 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.559.626 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.559.637 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.559.661 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.559.675 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.559.685 [graph_manager.cc:2810][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [227] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.559.732 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.559.744 [graph_manager.cc:2821][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [51] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.559.772 [graph_manager.cc:1087][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1447] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.560.350 [graph_manager.cc:1088][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [563] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.560.413 [graph_manager.cc:1089][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.560.439 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.560.459 [graph_manager.cc:1097][EVENT]167130 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:32.560.485 [graph_manager.cc:3325][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.570.238 [engine_place.cc:144][EVENT]167130 Run:The time cost of AIcoreEngine::CheckSupported is [9493] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.570.273 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.570.283 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.570.379 [graph_manager.cc:3351][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [9881] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.570.399 [graph_manager.cc:3364][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.570.480 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [29] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.570.525 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.570.704 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [168] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.570.750 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.570.799 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.570.839 [graph_manager.cc:3405][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [427] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.570.861 [graph_manager.cc:3412][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.762.084 [graph_manager.cc:3422][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [191207] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.762.139 [graph_manager.cc:3428][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.762.331 [graph_manager.cc:3467][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [169] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.762.352 [graph_manager.cc:3377][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [191941] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.762.370 [graph_manager.cc:1106][EVENT]167130 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [201894] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.762.383 [graph_manager.cc:1115][EVENT]167130 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:32.762.409 [graph_manager.cc:1130][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.762.444 [graph_manager.cc:1131][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.762.474 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.762.496 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.762.505 [graph_manager.cc:2837][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [45] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.762.652 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [28] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.762.665 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [5] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.762.675 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.762.700 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of BitcastPass is [3] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.762.709 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [9] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.762.717 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [18] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:32.762.727 [graph_manager.cc:2864][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [204] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.762.740 [graph_manager.cc:2872][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.762.762 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.762.777 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.762.795 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.762.810 [compile_nodes_pass.cc:88][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.762.821 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.762.831 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.762.945 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [104] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.762.998 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [40] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.763.013 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.763.028 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.763.043 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.763.052 [graph_manager.cc:2927][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [295] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.763.065 [graph_manager.cc:2937][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.763.081 [graph_manager.cc:2943][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.763.092 [graph_manager.cc:2950][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.763.300 [graph_manager.cc:2958][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [60] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.763.346 [graph_manager.cc:1132][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [888] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.763.434 [graph_manager.cc:1135][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [74] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.763.473 [graph_manager.cc:2975][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.763.504 [graph_manager.cc:2981][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.763.518 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.763.528 [graph_manager.cc:2986][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.763.537 [graph_manager.cc:1136][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [87] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.763.902 [graph_manager.cc:3555][EVENT]167130 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [323] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.764.040 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [34] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.764.074 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.764.226 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [140] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.764.263 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.764.309 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [34] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.764.336 [graph_builder.cc:865][EVENT]167130 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [362] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:32.764.745 [logger.cc:1071] 167130 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:37:32.764.777 [task_generator.cc:804][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [88] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.764.867 [task_generator.cc:805][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [77] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.766.898 [task_generator.cc:814][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [2016] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.766.918 [task_generator.cc:954][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2229] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.766.994 [task_generator.cc:967][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [41] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:32.767.021 [logger.cc:1084] 167130 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:37:32.767.972 [graph_manager.cc:1152][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4399] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.768.019 [graph_manager.cc:1164][EVENT]167130 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:32.768.057 [graph_manager.cc:1271][EVENT]167130 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [241274] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.768.070 [graph_manager.cc:1272][EVENT]167130 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:37:32.768.391 [atrace_api.c:93](tid:167130) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:37:32.768.407 [atrace_api.c:95](tid:167130) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:37:32.795.238 [graph_converter.cc:838][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [9912] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.795.472 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [174] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.797.174 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [1673] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.797.588 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [383] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.797.615 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [412] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.797.908 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [279] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.798.001 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [66] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.798.078 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [55] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.798.572 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [476] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.798.810 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [206] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.798.837 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [236] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.798.909 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [59] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.798.976 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [49] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.799.042 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [50] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.799.284 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [229] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.799.497 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [193] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.799.520 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [216] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.799.590 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [57] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.799.654 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [49] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.799.673 [graph_converter.cc:849][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4383] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.800.464 [graph_converter.cc:853][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [766] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.802.635 [graph_converter.cc:857][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2143] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.803.074 [graph_converter.cc:862][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [404] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.883.435 [graph_var_manager.cc:1424][EVENT]167129 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:32.883.537 [graph_manager.cc:1248][EVENT]167129 PreRun:PreRun start: graph node size 3, session id 5, graph id 4, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:32.883.828 [atrace_api.c:28](tid:167129) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:32.883.862 [trace_rb_log.c:84](tid:167129) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:32.883.878 [atrace_api.c:32](tid:167129) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:32.883.896 [client_manager.cpp:157][SetProfilingCallback][tid:167129] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:32.884.258 [parallel_partitioner.cc:165][EVENT]167129 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.884.295 [parallel_partitioner.cc:178][EVENT]167129 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.884.344 [graph_prepare.cc:1378][EVENT]167129 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.884.484 [graph_manager.cc:1050][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [157] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.884.513 [graph_manager.cc:1052][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.884.638 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.884.670 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.884.718 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.884.731 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.884.779 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.884.795 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.884.814 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.884.915 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.884.966 [graph_manager.cc:1054][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [441] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.885.193 [graph_manager.cc:1055][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [210] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.886.186 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.886.217 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.886.229 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.886.238 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [281] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.886.247 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.886.256 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.886.264 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [70] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.886.273 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.886.282 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.888.310 [graph_manager.cc:1056][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3095] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.888.377 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.888.397 [graph_prepare.cc:1982][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [50] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.888.764 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.888.790 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.888.801 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.888.810 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [179] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.888.819 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.888.828 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.888.836 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.888.845 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.888.853 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.888.906 [graph_prepare.cc:1983][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [495] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.888.931 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.888.942 [graph_prepare.cc:1984][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.888.956 [graph_prepare.cc:1985][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.888.971 [graph_prepare.cc:1986][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.888.982 [graph_prepare.cc:1987][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.888.997 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.889.009 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.889.023 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.889.107 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.889.118 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.889.127 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.889.136 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.889.144 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.889.153 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.889.161 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.889.169 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.889.178 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.889.186 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.889.194 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.889.202 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.889.210 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.889.227 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.889.236 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.889.245 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.889.267 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.889.280 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.889.309 [graph_prepare.cc:1988][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [318] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.889.322 [graph_manager.cc:1065][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [977] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.901.267 [graph_manager.cc:1077][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11926] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.901.337 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.901.386 [graph_manager.cc:1080][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [81] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.904.919 [graph_manager.cc:1081][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3518] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.904.962 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.904.976 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.904.987 [graph_manager.cc:1082][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [33] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.018 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.032 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.046 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.080 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [25] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.094 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.107 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.120 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.157 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.186 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.204 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.230 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.244 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.256 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.265 [graph_manager.cc:2700][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [252] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.370 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [0] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.905.383 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AddNPass is [0] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.905.392 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [0] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.905.401 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.905.409 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.905.418 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CastRemovePass is [8] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.905.426 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.905.434 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.905.442 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.905.451 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.905.459 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [6] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.905.467 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.905.476 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.905.484 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.905.492 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.905.501 [graph_manager.cc:2741][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [217] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.517 [graph_manager.cc:2752][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.540 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.551 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.568 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.584 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.595 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.607 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.625 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.638 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.652 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.663 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.676 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.717 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [33] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.739 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.753 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.762 [graph_manager.cc:2810][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [226] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.789 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.905.800 [graph_manager.cc:2821][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [29] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.829 [graph_manager.cc:1087][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [823] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.960 [graph_manager.cc:1088][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [118] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.905.996 [graph_manager.cc:1089][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.906.014 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.906.028 [graph_manager.cc:1097][EVENT]167129 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:32.906.058 [graph_manager.cc:3325][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.906.416 [engine_place.cc:144][EVENT]167129 Run:The time cost of AIcoreEngine::CheckSupported is [257] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.906.443 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.906.452 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.906.524 [graph_manager.cc:3351][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [453] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.906.543 [graph_manager.cc:3364][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.906.608 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.906.625 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.906.755 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [121] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.906.796 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.906.843 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.906.875 [graph_manager.cc:3405][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [319] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.906.893 [graph_manager.cc:3412][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.915.774 [graph_manager.cc:3422][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [8868] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.915.807 [graph_manager.cc:3428][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.915.928 [graph_manager.cc:3467][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [100] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.915.946 [graph_manager.cc:3377][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [9391] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.915.962 [graph_manager.cc:1106][EVENT]167129 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [9911] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.915.974 [graph_manager.cc:1115][EVENT]167129 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:32.915.997 [graph_manager.cc:1130][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.040 [graph_manager.cc:1131][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.064 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.082 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.091 [graph_manager.cc:2837][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.160 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.916.172 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.916.182 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.916.191 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.916.199 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.916.208 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.916.218 [graph_manager.cc:2864][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [110] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.230 [graph_manager.cc:2872][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.248 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.262 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.277 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.291 [compile_nodes_pass.cc:88][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.301 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.310 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.382 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [62] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.407 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.421 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.442 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.454 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.464 [graph_manager.cc:2927][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [219] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.476 [graph_manager.cc:2937][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.489 [graph_manager.cc:2943][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.500 [graph_manager.cc:2950][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.660 [graph_manager.cc:2958][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.690 [graph_manager.cc:1132][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [637] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.755 [graph_manager.cc:1135][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [52] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.786 [graph_manager.cc:2975][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.816 [graph_manager.cc:2981][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.830 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.840 [graph_manager.cc:2986][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.849 [graph_manager.cc:1136][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [77] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.916.958 [graph_manager.cc:3555][EVENT]167129 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [80] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.917.039 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.917.055 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.917.145 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [80] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.917.174 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.917.211 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [26] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.917.231 [graph_builder.cc:865][EVENT]167129 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [220] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:32.917.520 [logger.cc:1071] 167129 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:37:32.917.549 [task_generator.cc:804][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [77] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.917.606 [task_generator.cc:805][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [45] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.918.332 [task_generator.cc:814][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [713] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.918.348 [task_generator.cc:954][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [876] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.918.408 [task_generator.cc:967][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [34] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:32.918.426 [logger.cc:1084] 167129 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:37:32.918.594 [graph_manager.cc:1152][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [1722] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.918.613 [graph_manager.cc:1164][EVENT]167129 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:32.918.645 [graph_manager.cc:1271][EVENT]167129 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [34473] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.918.656 [graph_manager.cc:1272][EVENT]167129 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:37:32.918.966 [atrace_api.c:93](tid:167129) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:37:32.918.981 [atrace_api.c:95](tid:167129) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:37:32.923.737 [graph_converter.cc:838][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1329] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.923.899 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [116] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.924.360 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [435] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.924.550 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [166] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.924.572 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [189] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.924.787 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [202] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.924.824 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.924.853 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.925.037 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [173] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.925.116 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [61] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.925.130 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [75] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.925.157 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.925.192 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.925.218 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.925.289 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [62] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.925.354 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [54] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.925.366 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [65] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.925.390 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.925.413 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.925.426 [graph_converter.cc:849][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1650] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.925.631 [graph_converter.cc:853][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [195] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.926.293 [graph_converter.cc:857][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [646] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.926.431 [graph_converter.cc:862][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [110] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.991.551 [graph_var_manager.cc:1424][EVENT]167130 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:32.991.652 [graph_manager.cc:1248][EVENT]167130 PreRun:PreRun start: graph node size 3, session id 6, graph id 5, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:32.991.928 [atrace_api.c:28](tid:167130) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:32.991.957 [trace_rb_log.c:84](tid:167130) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:32.991.970 [atrace_api.c:32](tid:167130) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:32.991.988 [client_manager.cpp:157][SetProfilingCallback][tid:167130] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:32.992.378 [parallel_partitioner.cc:165][EVENT]167130 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.992.415 [parallel_partitioner.cc:178][EVENT]167130 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.992.460 [graph_prepare.cc:1378][EVENT]167130 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.992.631 [graph_manager.cc:1050][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [186] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.992.654 [graph_manager.cc:1052][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.992.779 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.992.836 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.992.884 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.992.897 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.992.944 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.992.957 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.992.974 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.993.075 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.993.095 [graph_manager.cc:1054][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [428] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.993.321 [graph_manager.cc:1055][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [213] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.994.338 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.994.369 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.994.379 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.994.389 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [342] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.994.398 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.994.407 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.994.415 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.994.424 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.994.432 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.996.557 [graph_manager.cc:1056][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3214] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.996.621 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.996.639 [graph_prepare.cc:1982][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [49] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.997.060 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.997.097 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.108 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.118 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [206] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.126 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.135 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:32.997.143 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.152 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.160 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.202 [graph_prepare.cc:1983][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [549] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.997.226 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.997.238 [graph_prepare.cc:1984][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.997.251 [graph_prepare.cc:1985][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.997.266 [graph_prepare.cc:1986][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.997.277 [graph_prepare.cc:1987][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.997.291 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.997.303 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.997.317 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.997.399 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.411 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.420 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.429 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.437 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.446 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.462 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.470 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.479 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.487 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.495 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.504 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.512 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.520 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.528 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.537 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:32.997.559 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.997.572 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.997.603 [graph_prepare.cc:1988][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [317] micro second. [INFO] GE(164040,python):2024-01-10-11:37:32.997.616 [graph_manager.cc:1065][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1025] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.009.628 [graph_manager.cc:1077][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11992] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.009.707 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.009.760 [graph_manager.cc:1080][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [94] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.015.151 [graph_manager.cc:1081][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [5375] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.015.196 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.015.212 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.015.225 [graph_manager.cc:1082][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.015.257 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.015.284 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.015.299 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.015.442 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [132] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.015.460 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.015.538 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [68] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.015.554 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.015.602 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.015.623 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.015.642 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.015.721 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [69] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.015.739 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.015.753 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.015.763 [graph_manager.cc:2700][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [511] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.015.970 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.015.984 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.015.994 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.016.003 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.016.012 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.016.021 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CastRemovePass is [34] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.016.029 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.016.038 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.016.046 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [10] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.016.064 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.016.073 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.016.082 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.016.090 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.016.098 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [8] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.016.106 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.016.116 [graph_manager.cc:2741][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [334] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.016.125 [graph_manager.cc:2752][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.016.149 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.016.162 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.016.183 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.016.198 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.016.211 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.016.223 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.016.243 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.016.259 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.016.272 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.016.282 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.016.294 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.016.306 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.016.326 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.016.339 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.016.348 [graph_manager.cc:2810][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [203] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.016.394 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.016.407 [graph_manager.cc:2821][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [44] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.016.437 [graph_manager.cc:1087][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1193] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.016.925 [graph_manager.cc:1088][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [474] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.016.983 [graph_manager.cc:1089][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.017.008 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.017.025 [graph_manager.cc:1097][EVENT]167130 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:33.017.046 [graph_manager.cc:3325][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.024.436 [engine_place.cc:144][EVENT]167130 Run:The time cost of AIcoreEngine::CheckSupported is [7203] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.024.467 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.024.478 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.024.566 [graph_manager.cc:3351][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [7506] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.024.585 [graph_manager.cc:3364][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.024.665 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [24] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.024.693 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.024.838 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [134] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.024.881 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [29] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.024.928 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.024.963 [graph_manager.cc:3405][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [365] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.024.981 [graph_manager.cc:3412][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.096.447 [graph_manager.cc:3422][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [71451] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.096.508 [graph_manager.cc:3428][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.096.663 [graph_manager.cc:3467][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [133] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.096.682 [graph_manager.cc:3377][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [72086] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.096.699 [graph_manager.cc:1106][EVENT]167130 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [79660] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.096.713 [graph_manager.cc:1115][EVENT]167130 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:33.096.738 [graph_manager.cc:1130][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.096.770 [graph_manager.cc:1131][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.096.799 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.096.818 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.096.828 [graph_manager.cc:2837][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [42] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.096.948 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [21] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.096.962 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [5] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.096.971 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.096.980 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of BitcastPass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.096.989 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [8] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.096.998 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [15] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:37:33.097.007 [graph_manager.cc:2864][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [161] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.020 [graph_manager.cc:2872][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.040 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.055 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.071 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.095 [compile_nodes_pass.cc:88][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.105 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.115 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.209 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [85] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.245 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [24] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.259 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.273 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.288 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.298 [graph_manager.cc:2927][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [262] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.310 [graph_manager.cc:2937][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.324 [graph_manager.cc:2943][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.335 [graph_manager.cc:2950][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.525 [graph_manager.cc:2958][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [49] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.560 [graph_manager.cc:1132][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [776] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.639 [graph_manager.cc:1135][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [65] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.675 [graph_manager.cc:2975][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.767 [graph_manager.cc:2981][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.784 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.794 [graph_manager.cc:2986][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.097.804 [graph_manager.cc:1136][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [149] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.098.087 [graph_manager.cc:3555][EVENT]167130 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [245] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.098.218 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.098.243 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.098.354 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [101] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.098.387 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.098.428 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [29] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.098.453 [graph_builder.cc:865][EVENT]167130 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [288] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:33.098.797 [logger.cc:1071] 167130 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:37:33.098.829 [task_generator.cc:804][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [76] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.098.903 [task_generator.cc:805][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [62] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.100.301 [task_generator.cc:814][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1382] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.100.317 [task_generator.cc:954][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1565] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.100.381 [task_generator.cc:967][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [35] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:33.100.402 [logger.cc:1084] 167130 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:37:33.101.193 [graph_manager.cc:1152][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3358] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.101.230 [graph_manager.cc:1164][EVENT]167130 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:33.101.269 [graph_manager.cc:1271][EVENT]167130 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [108975] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.101.280 [graph_manager.cc:1272][EVENT]167130 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:37:33.101.598 [atrace_api.c:93](tid:167130) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:37:33.101.614 [atrace_api.c:95](tid:167130) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:37:33.119.579 [graph_converter.cc:838][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [6563] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.119.790 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [152] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.120.954 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [1134] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.121.240 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [255] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.121.281 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [297] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.121.523 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [228] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.121.589 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [43] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.121.643 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.122.042 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [385] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.122.210 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [140] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.122.232 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [163] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.122.283 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [40] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.122.329 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [33] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.122.375 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.122.543 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [155] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.122.686 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [127] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.122.701 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [143] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.122.748 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.122.792 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.122.808 [graph_converter.cc:849][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [3176] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.123.329 [graph_converter.cc:853][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [510] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.124.809 [graph_converter.cc:857][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1456] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.125.111 [graph_converter.cc:862][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [272] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.194.694 [graph_var_manager.cc:1424][EVENT]167129 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:33.194.795 [graph_manager.cc:1248][EVENT]167129 PreRun:PreRun start: graph node size 4, session id 7, graph id 6, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:33.195.074 [atrace_api.c:28](tid:167129) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:33.195.105 [trace_rb_log.c:84](tid:167129) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:33.195.145 [atrace_api.c:32](tid:167129) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:33.195.163 [client_manager.cpp:157][SetProfilingCallback][tid:167129] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:33.195.540 [parallel_partitioner.cc:165][EVENT]167129 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.195.576 [parallel_partitioner.cc:178][EVENT]167129 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.195.623 [graph_prepare.cc:1378][EVENT]167129 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.195.754 [graph_manager.cc:1050][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [149] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.195.779 [graph_manager.cc:1052][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.195.917 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.195.950 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.196.000 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.196.013 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.196.059 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.196.072 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.196.090 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.196.190 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.196.211 [graph_manager.cc:1054][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [419] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.196.433 [graph_manager.cc:1055][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [209] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.197.621 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:33.197.651 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.197.662 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.197.672 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [471] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.197.681 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.197.709 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [4] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:33.197.729 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.197.739 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.197.748 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.200.619 [graph_manager.cc:1056][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [4164] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.200.689 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.200.708 [graph_prepare.cc:1982][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.201.246 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:33.201.274 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.285 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.294 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [340] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.304 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.313 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:33.201.321 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.330 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.339 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.363 [graph_prepare.cc:1983][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [643] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.201.387 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.201.398 [graph_prepare.cc:1984][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.201.412 [graph_prepare.cc:1985][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.201.427 [graph_prepare.cc:1986][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.201.438 [graph_prepare.cc:1987][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.201.452 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.201.476 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.201.491 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.201.585 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.597 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.606 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.614 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.623 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.631 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.640 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.648 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.657 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.665 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.673 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.681 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.707 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.716 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.724 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.732 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.201.756 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.201.768 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.201.801 [graph_prepare.cc:1988][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [355] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.201.814 [graph_manager.cc:1065][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1161] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.214.784 [graph_manager.cc:1077][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12949] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.214.894 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.214.946 [graph_manager.cc:1080][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [113] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.225.991 [graph_manager.cc:1081][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [11027] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.226.036 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.226.052 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.226.064 [graph_manager.cc:1082][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.226.097 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.226.114 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.226.128 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.226.285 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [146] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.226.307 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.226.400 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [83] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.226.420 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.226.473 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.226.499 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.226.521 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.226.614 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [78] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.226.636 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.226.650 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.226.660 [graph_manager.cc:2700][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [570] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.226.900 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [4] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.226.918 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.226.939 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.226.949 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [5] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.226.957 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.226.966 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CastRemovePass is [41] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.226.975 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [5] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.226.983 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.226.992 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [12] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.227.004 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.227.012 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [18] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.227.020 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.227.029 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.227.037 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [9] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.227.045 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.227.059 [graph_manager.cc:2741][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [379] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.227.071 [graph_manager.cc:2752][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.227.100 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.227.116 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.227.143 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.227.161 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.227.177 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.227.190 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.227.211 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.227.231 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.227.245 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.227.255 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.227.267 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.227.279 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.227.300 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.227.314 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.227.322 [graph_manager.cc:2810][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [228] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.227.366 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.227.381 [graph_manager.cc:2821][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [51] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.227.410 [graph_manager.cc:1087][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1326] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.227.965 [graph_manager.cc:1088][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [538] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.228.029 [graph_manager.cc:1089][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.228.053 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.228.071 [graph_manager.cc:1097][EVENT]167129 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:33.228.095 [graph_manager.cc:3325][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.237.635 [engine_place.cc:144][EVENT]167129 Run:The time cost of AIcoreEngine::CheckSupported is [9311] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.237.669 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.237.679 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.237.783 [graph_manager.cc:3351][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [9671] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.237.803 [graph_manager.cc:3364][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.237.881 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [29] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.237.927 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.238.100 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [161] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.238.144 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.238.193 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.238.229 [graph_manager.cc:3405][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [412] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.238.247 [graph_manager.cc:3412][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.364.309 [graph_manager.cc:3422][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [126046] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.364.366 [graph_manager.cc:3428][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.364.563 [graph_manager.cc:3467][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [171] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.364.583 [graph_manager.cc:3377][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [126768] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.364.603 [graph_manager.cc:1106][EVENT]167129 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [136516] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.364.617 [graph_manager.cc:1115][EVENT]167129 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:33.364.643 [graph_manager.cc:1130][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.364.677 [graph_manager.cc:1131][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.364.707 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.364.728 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.364.739 [graph_manager.cc:2837][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [44] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.364.883 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [29] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.364.898 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.364.908 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.364.917 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.364.942 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.364.951 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [15] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.364.961 [graph_manager.cc:2864][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [204] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.364.974 [graph_manager.cc:2872][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.364.995 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.011 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.028 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.044 [compile_nodes_pass.cc:88][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.054 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.063 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.177 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [104] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.232 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [42] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.246 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.261 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.277 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.287 [graph_manager.cc:2927][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [296] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.300 [graph_manager.cc:2937][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.315 [graph_manager.cc:2943][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.326 [graph_manager.cc:2950][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.535 [graph_manager.cc:2958][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [56] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.569 [graph_manager.cc:1132][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [877] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.666 [graph_manager.cc:1135][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [74] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.722 [graph_manager.cc:2975][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.756 [graph_manager.cc:2981][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.771 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.782 [graph_manager.cc:2986][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.365.792 [graph_manager.cc:1136][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [109] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.366.132 [graph_manager.cc:3555][EVENT]167129 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [299] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.366.268 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.366.298 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.366.452 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [143] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.366.488 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.366.534 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.366.560 [graph_builder.cc:865][EVENT]167129 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [356] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:33.367.045 [logger.cc:1071] 167129 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:37:33.367.081 [task_generator.cc:804][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [142] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.367.170 [task_generator.cc:805][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [76] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.368.966 [task_generator.cc:814][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1781] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.368.982 [task_generator.cc:954][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2043] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.369.056 [task_generator.cc:967][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [39] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:33.369.082 [logger.cc:1084] 167129 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:37:33.371.449 [graph_manager.cc:1152][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [5623] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.371.508 [graph_manager.cc:1164][EVENT]167129 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:33.371.558 [graph_manager.cc:1271][EVENT]167129 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [176107] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.371.572 [graph_manager.cc:1272][EVENT]167129 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:37:33.371.987 [atrace_api.c:93](tid:167129) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:37:33.372.007 [atrace_api.c:95](tid:167129) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:37:33.402.638 [graph_converter.cc:838][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [12244] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.402.870 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [176] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.404.534 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [1636] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.404.947 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [382] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.404.976 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [413] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.405.251 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [261] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.405.339 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [63] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.405.409 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [50] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.405.895 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [470] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.406.119 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [195] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.406.142 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [218] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.406.208 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [55] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.406.269 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [46] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.406.329 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [47] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.406.557 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [215] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.406.754 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [178] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.406.773 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [197] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.406.835 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [53] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.406.895 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [46] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.406.914 [graph_converter.cc:849][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4226] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.407.648 [graph_converter.cc:853][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [707] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.409.680 [graph_converter.cc:857][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2003] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.410.102 [graph_converter.cc:862][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [381] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.492.550 [graph_var_manager.cc:1424][EVENT]167130 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:33.492.652 [graph_manager.cc:1248][EVENT]167130 PreRun:PreRun start: graph node size 4, session id 8, graph id 7, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:33.493.112 [atrace_api.c:28](tid:167130) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:33.493.168 [trace_rb_log.c:84](tid:167130) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:33.493.181 [atrace_api.c:32](tid:167130) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:33.493.202 [client_manager.cpp:157][SetProfilingCallback][tid:167130] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:33.493.618 [parallel_partitioner.cc:165][EVENT]167130 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.493.657 [parallel_partitioner.cc:178][EVENT]167130 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.493.761 [graph_prepare.cc:1378][EVENT]167130 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.493.919 [graph_manager.cc:1050][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [177] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.493.942 [graph_manager.cc:1052][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.494.085 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.494.116 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.494.166 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.494.179 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.494.229 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.494.242 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.494.259 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.494.369 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.494.390 [graph_manager.cc:1054][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [434] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.494.644 [graph_manager.cc:1055][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [215] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.495.759 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:33.495.788 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.495.799 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.495.809 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [340] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.495.818 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.495.827 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [4] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:33.495.835 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [67] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.495.844 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.495.852 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.498.107 [graph_manager.cc:1056][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3442] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.498.177 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.498.196 [graph_prepare.cc:1982][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [54] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.498.691 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:33.498.718 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.498.729 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.498.738 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [250] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.498.747 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.498.756 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:33.498.765 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.498.773 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.498.782 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.498.851 [graph_prepare.cc:1983][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [641] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.498.880 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.498.891 [graph_prepare.cc:1984][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.498.905 [graph_prepare.cc:1985][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.498.920 [graph_prepare.cc:1986][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.498.931 [graph_prepare.cc:1987][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.498.946 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.498.957 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.498.970 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.499.065 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.499.077 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.499.086 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.499.095 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.499.103 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.499.112 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.499.120 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.499.128 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.499.137 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of StopGradientPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.499.145 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.499.153 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.499.162 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.499.170 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.499.178 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.499.195 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.499.204 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.499.228 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.499.241 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.499.275 [graph_prepare.cc:1988][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [335] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.499.287 [graph_manager.cc:1065][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1146] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.511.689 [graph_manager.cc:1077][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12381] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.511.818 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.511.871 [graph_manager.cc:1080][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [143] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.518.569 [graph_manager.cc:1081][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [6681] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.518.614 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.518.630 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.518.642 [graph_manager.cc:1082][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.518.674 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.518.689 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.518.704 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.518.737 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.518.750 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.518.764 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.518.777 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.518.818 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.518.860 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.518.879 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.518.907 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.518.922 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.518.935 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.518.946 [graph_manager.cc:2700][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [278] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.069 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [0] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.519.082 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AddNPass is [0] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.519.092 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.519.101 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.519.110 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.519.118 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.519.127 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.519.135 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.519.143 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.519.151 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.519.160 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.519.168 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.519.176 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.519.184 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.519.193 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.519.202 [graph_manager.cc:2741][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [238] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.211 [graph_manager.cc:2752][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.241 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.253 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.271 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.286 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.298 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.311 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.331 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.345 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.358 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.367 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.380 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.390 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.409 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.424 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.433 [graph_manager.cc:2810][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [197] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.463 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.519.475 [graph_manager.cc:2821][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [33] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.503 [graph_manager.cc:1087][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [843] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.638 [graph_manager.cc:1088][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [122] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.678 [graph_manager.cc:1089][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.697 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.519.713 [graph_manager.cc:1097][EVENT]167130 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:33.519.743 [graph_manager.cc:3325][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.520.124 [engine_place.cc:144][EVENT]167130 Run:The time cost of AIcoreEngine::CheckSupported is [277] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.520.151 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.520.160 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.520.240 [graph_manager.cc:3351][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [486] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.520.258 [graph_manager.cc:3364][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.520.330 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.520.348 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.520.514 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [157] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.520.561 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.520.609 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.520.645 [graph_manager.cc:3405][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [375] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.520.662 [graph_manager.cc:3412][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.562.683 [graph_manager.cc:3422][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [42005] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.562.734 [graph_manager.cc:3428][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.562.894 [graph_manager.cc:3467][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [139] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.562.915 [graph_manager.cc:3377][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [42645] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.562.931 [graph_manager.cc:1106][EVENT]167130 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [43196] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.562.945 [graph_manager.cc:1115][EVENT]167130 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:33.562.970 [graph_manager.cc:1130][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.016 [graph_manager.cc:1131][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.043 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.061 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.071 [graph_manager.cc:2837][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.164 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.563.177 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.563.186 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.563.195 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.563.204 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.563.212 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.563.222 [graph_manager.cc:2864][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [134] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.234 [graph_manager.cc:2872][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.255 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.269 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.286 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.300 [compile_nodes_pass.cc:88][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.310 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.320 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.405 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [75] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.435 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.448 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.462 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.483 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.492 [graph_manager.cc:2927][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [242] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.505 [graph_manager.cc:2937][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.518 [graph_manager.cc:2943][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.530 [graph_manager.cc:2950][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.715 [graph_manager.cc:2958][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [40] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.748 [graph_manager.cc:1132][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [717] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.823 [graph_manager.cc:1135][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [62] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.857 [graph_manager.cc:2975][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.894 [graph_manager.cc:2981][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.909 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.920 [graph_manager.cc:2986][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.563.929 [graph_manager.cc:1136][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [91] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.564.059 [graph_manager.cc:3555][EVENT]167130 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [96] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.564.158 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.564.176 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.564.306 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [121] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.564.343 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [24] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.564.389 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [34] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.564.415 [graph_builder.cc:865][EVENT]167130 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [294] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:33.564.740 [logger.cc:1071] 167130 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:37:33.564.771 [task_generator.cc:804][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [85] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.564.838 [task_generator.cc:805][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [54] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.565.612 [task_generator.cc:814][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [758] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.565.626 [task_generator.cc:954][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [941] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.565.737 [task_generator.cc:967][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [85] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:33.565.758 [logger.cc:1084] 167130 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:37:33.566.388 [graph_manager.cc:1152][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2433] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.566.422 [graph_manager.cc:1164][EVENT]167130 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:33.566.459 [graph_manager.cc:1271][EVENT]167130 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [72935] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.566.471 [graph_manager.cc:1272][EVENT]167130 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:37:33.566.792 [atrace_api.c:93](tid:167130) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:37:33.566.808 [atrace_api.c:95](tid:167130) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:37:33.579.817 [graph_converter.cc:838][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [4209] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.579.994 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [129] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.580.527 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [508] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.580.758 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [204] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.580.781 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [229] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.581.005 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [211] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.581.049 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [24] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.581.081 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.581.283 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [189] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.581.373 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [67] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.581.391 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [86] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.581.421 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.581.463 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.581.493 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.581.576 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [70] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.581.648 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [58] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.581.662 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [72] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.581.734 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [59] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.581.764 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.581.781 [graph_converter.cc:849][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1922] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.582.017 [graph_converter.cc:853][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [224] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.582.805 [graph_converter.cc:857][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [769] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.582.963 [graph_converter.cc:862][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [129] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.672.302 [graph_var_manager.cc:1424][EVENT]167131 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:33.672.408 [graph_manager.cc:1248][EVENT]167131 PreRun:PreRun start: graph node size 3, session id 9, graph id 8, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:33.673.327 [atrace_api.c:28](tid:167131) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:33.673.406 [trace_rb_log.c:84](tid:167131) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:33.673.421 [atrace_api.c:32](tid:167131) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:33.673.439 [client_manager.cpp:157][SetProfilingCallback][tid:167131] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:33.674.341 [parallel_partitioner.cc:165][EVENT]167131 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.674.382 [parallel_partitioner.cc:178][EVENT]167131 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.674.428 [graph_prepare.cc:1378][EVENT]167131 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.675.060 [graph_manager.cc:1050][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [647] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.675.091 [graph_manager.cc:1052][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.675.216 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.675.270 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.675.320 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.675.333 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.675.383 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.675.396 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.675.413 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.675.515 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.675.540 [graph_manager.cc:1054][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [435] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.675.765 [graph_manager.cc:1055][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [211] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.676.648 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:33.676.677 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.676.689 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.676.698 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of InferShapePass is [268] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.676.707 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.676.716 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:33.676.725 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.676.733 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.676.742 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.678.784 [graph_manager.cc:1056][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2995] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.678.851 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.678.871 [graph_prepare.cc:1982][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [50] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.679.204 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:33.679.241 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.252 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.262 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of InferShapePass is [170] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.271 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.280 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:33.679.288 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.297 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.305 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of InferValuePass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.329 [graph_prepare.cc:1983][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [443] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.679.354 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.679.365 [graph_prepare.cc:1984][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.679.379 [graph_prepare.cc:1985][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.679.397 [graph_prepare.cc:1986][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.679.409 [graph_prepare.cc:1987][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.679.424 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.679.436 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.679.450 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.679.533 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.546 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.555 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.564 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.572 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.581 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.609 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.618 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.627 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.635 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.644 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.652 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.660 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.669 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.677 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.685 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.679.708 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.679.721 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.679.750 [graph_prepare.cc:1988][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [332] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.679.764 [graph_manager.cc:1065][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [944] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.691.693 [graph_manager.cc:1077][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11908] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.691.764 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.691.814 [graph_manager.cc:1080][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [81] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.341 [graph_manager.cc:1081][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3511] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.384 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.398 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.410 [graph_manager.cc:1082][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [33] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.440 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.465 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.478 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.506 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.520 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.534 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.546 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.583 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.601 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.619 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.643 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.658 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.670 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.680 [graph_manager.cc:2700][EVENT]167131 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [244] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.784 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of EnterPass is [0] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.695.797 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.695.807 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.695.816 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.695.824 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.695.833 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of CastRemovePass is [8] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.695.841 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.695.850 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.695.858 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.695.867 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.695.882 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [6] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.695.891 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.695.900 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.695.908 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.695.916 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.695.926 [graph_manager.cc:2741][EVENT]167131 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [227] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.935 [graph_manager.cc:2752][EVENT]167131 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.957 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.969 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.695.985 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.000 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.011 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.023 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.040 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.054 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.067 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.077 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.089 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.100 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.117 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.129 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.137 [graph_manager.cc:2810][EVENT]167131 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [184] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.169 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.696.180 [graph_manager.cc:2821][EVENT]167131 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [29] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.207 [graph_manager.cc:1087][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [780] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.341 [graph_manager.cc:1088][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [121] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.378 [graph_manager.cc:1089][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.396 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.410 [graph_manager.cc:1097][EVENT]167131 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:33.696.430 [graph_manager.cc:3325][EVENT]167131 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.801 [engine_place.cc:144][EVENT]167131 Run:The time cost of AIcoreEngine::CheckSupported is [278] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.829 [engine_place.cc:144][EVENT]167131 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.838 [engine_place.cc:144][EVENT]167131 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.910 [graph_manager.cc:3351][EVENT]167131 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [467] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.928 [graph_manager.cc:3364][EVENT]167131 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.696.988 [engine_partitioner.cc:1139][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.697.004 [engine_partitioner.cc:1142][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.697.131 [engine_partitioner.cc:1148][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [117] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.697.172 [engine_partitioner.cc:1155][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.697.225 [engine_partitioner.cc:1164][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [41] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.697.262 [graph_manager.cc:3405][EVENT]167131 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [321] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.697.280 [graph_manager.cc:3412][EVENT]167131 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.706.824 [graph_manager.cc:3422][EVENT]167131 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [9531] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.706.872 [graph_manager.cc:3428][EVENT]167131 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.706.994 [graph_manager.cc:3467][EVENT]167131 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [101] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.013 [graph_manager.cc:3377][EVENT]167131 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [10073] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.028 [graph_manager.cc:1106][EVENT]167131 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [10604] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.041 [graph_manager.cc:1115][EVENT]167131 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:33.707.063 [graph_manager.cc:1130][EVENT]167131 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.093 [graph_manager.cc:1131][EVENT]167131 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.117 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.134 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.144 [graph_manager.cc:2837][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.212 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.707.225 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.707.234 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of CondRemovePass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.707.243 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.707.251 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.707.260 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:33.707.269 [graph_manager.cc:2864][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [110] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.280 [graph_manager.cc:2872][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.300 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.314 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.328 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.351 [compile_nodes_pass.cc:88][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.362 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.372 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.444 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [64] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.472 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.485 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.498 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.511 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.521 [graph_manager.cc:2927][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [223] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.533 [graph_manager.cc:2937][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.546 [graph_manager.cc:2943][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.556 [graph_manager.cc:2950][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.719 [graph_manager.cc:2958][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.751 [graph_manager.cc:1132][EVENT]167131 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [644] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.817 [graph_manager.cc:1135][EVENT]167131 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [54] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.854 [graph_manager.cc:2975][EVENT]167131 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.885 [graph_manager.cc:2981][EVENT]167131 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.899 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.909 [graph_manager.cc:2986][EVENT]167131 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.707.918 [graph_manager.cc:1136][EVENT]167131 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [85] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.708.030 [graph_manager.cc:3555][EVENT]167131 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [82] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.708.123 [engine_partitioner.cc:1139][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.708.138 [engine_partitioner.cc:1142][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.708.241 [engine_partitioner.cc:1148][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [93] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.708.271 [engine_partitioner.cc:1155][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.708.310 [engine_partitioner.cc:1164][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.708.331 [graph_builder.cc:865][EVENT]167131 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [237] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:33.708.739 [logger.cc:1071] 167131 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:37:33.708.773 [task_generator.cc:804][EVENT]167131 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [167] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.708.830 [task_generator.cc:805][EVENT]167131 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [45] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.709.526 [task_generator.cc:814][EVENT]167131 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [681] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.709.541 [task_generator.cc:954][EVENT]167131 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [935] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.709.603 [task_generator.cc:967][EVENT]167131 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [35] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:33.709.621 [logger.cc:1084] 167131 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:37:33.709.841 [graph_manager.cc:1152][EVENT]167131 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [1899] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.709.861 [graph_manager.cc:1164][EVENT]167131 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:33.709.892 [graph_manager.cc:1271][EVENT]167131 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [35638] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.709.904 [graph_manager.cc:1272][EVENT]167131 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:37:33.710.211 [atrace_api.c:93](tid:167131) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:37:33.710.226 [atrace_api.c:95](tid:167131) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:37:33.714.803 [graph_converter.cc:838][EVENT]167131 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1290] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.714.967 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of ZeroCopy is [116] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.715.420 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of CEM is [429] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.715.614 [copy_flow_launch_fuse.cc:395][EVENT]167131 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [165] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.715.648 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [202] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.715.864 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [203] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.715.903 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.715.933 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.716.120 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of CEM is [176] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.716.201 [copy_flow_launch_fuse.cc:395][EVENT]167131 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [62] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.716.216 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [77] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.716.244 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.716.269 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.716.295 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.716.368 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of CEM is [62] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.716.434 [copy_flow_launch_fuse.cc:395][EVENT]167131 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [54] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.716.445 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [65] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.716.469 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.716.492 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.716.505 [graph_converter.cc:849][EVENT]167131 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1661] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.716.710 [graph_converter.cc:853][EVENT]167131 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [195] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.717.358 [graph_converter.cc:857][EVENT]167131 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [631] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.717.496 [graph_converter.cc:862][EVENT]167131 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [111] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.782.567 [graph_var_manager.cc:1424][EVENT]167130 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:33.782.671 [graph_manager.cc:1248][EVENT]167130 PreRun:PreRun start: graph node size 4, session id 10, graph id 9, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:33.782.937 [atrace_api.c:28](tid:167130) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:33.782.965 [trace_rb_log.c:84](tid:167130) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:33.782.978 [atrace_api.c:32](tid:167130) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:33.783.025 [client_manager.cpp:157][SetProfilingCallback][tid:167130] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:33.783.391 [parallel_partitioner.cc:165][EVENT]167130 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.783.428 [parallel_partitioner.cc:178][EVENT]167130 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.783.475 [graph_prepare.cc:1378][EVENT]167130 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.783.619 [graph_manager.cc:1050][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [161] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.783.642 [graph_manager.cc:1052][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.783.780 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.783.810 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.783.860 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.783.874 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.783.919 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.783.933 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.783.950 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.784.054 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.784.076 [graph_manager.cc:1054][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [421] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.784.300 [graph_manager.cc:1055][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [210] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.785.448 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:33.785.479 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.785.490 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.785.500 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [438] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.785.509 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [15] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.785.518 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [4] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:33.785.538 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.785.547 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.785.556 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.788.586 [graph_manager.cc:1056][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [4264] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.788.656 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.788.676 [graph_prepare.cc:1982][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.789.218 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:33.789.246 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.258 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.267 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [340] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.276 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.285 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:33.789.293 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.301 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.310 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.335 [graph_prepare.cc:1983][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [645] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.789.359 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.789.370 [graph_prepare.cc:1984][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.789.384 [graph_prepare.cc:1985][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.789.397 [graph_prepare.cc:1986][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.789.407 [graph_prepare.cc:1987][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.789.421 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.789.432 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.789.457 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.789.550 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.562 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.571 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.580 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.588 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.596 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.605 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.613 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.621 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of StopGradientPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.630 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.638 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.646 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.654 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.662 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.671 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.679 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:33.789.744 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.789.757 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.789.789 [graph_prepare.cc:1988][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [373] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.789.802 [graph_manager.cc:1065][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1180] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.802.807 [graph_manager.cc:1077][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12984] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.802.890 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.802.942 [graph_manager.cc:1080][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [86] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.813.967 [graph_manager.cc:1081][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [11008] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.814.015 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.814.032 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.814.045 [graph_manager.cc:1082][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.814.079 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.814.095 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.814.110 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.814.260 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [138] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.814.278 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.814.368 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [81] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.814.384 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.814.434 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.814.456 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.814.476 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.814.563 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [76] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.814.580 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.814.593 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.814.602 [graph_manager.cc:2700][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [530] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.814.840 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.814.856 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AddNPass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.814.876 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.814.886 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.814.895 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.814.904 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CastRemovePass is [42] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.814.912 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.814.921 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [5] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.814.929 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [10] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.814.938 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.814.946 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.814.954 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.814.962 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.814.970 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [10] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.814.978 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.814.988 [graph_manager.cc:2741][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [368] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.814.997 [graph_manager.cc:2752][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.021 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.034 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.056 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.072 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.085 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.098 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.119 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.135 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.155 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.167 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.180 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.193 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.214 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.229 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.239 [graph_manager.cc:2810][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [222] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.284 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.815.296 [graph_manager.cc:2821][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [48] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.324 [graph_manager.cc:1087][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1260] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.882 [graph_manager.cc:1088][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [542] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.944 [graph_manager.cc:1089][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [30] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.968 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.815.985 [graph_manager.cc:1097][EVENT]167130 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:33.816.007 [graph_manager.cc:3325][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.826.549 [engine_place.cc:144][EVENT]167130 Run:The time cost of AIcoreEngine::CheckSupported is [10324] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.826.584 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.826.594 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.826.688 [graph_manager.cc:3351][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10667] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.826.708 [graph_manager.cc:3364][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.826.784 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.826.815 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.826.996 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [160] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.827.038 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.827.090 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.827.126 [graph_manager.cc:3405][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [405] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.827.148 [graph_manager.cc:3412][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.942.373 [graph_manager.cc:3422][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [115207] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.942.442 [graph_manager.cc:3428][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.942.619 [graph_manager.cc:3467][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [154] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.942.639 [graph_manager.cc:3377][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [115920] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.942.656 [graph_manager.cc:1106][EVENT]167130 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [126657] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.942.669 [graph_manager.cc:1115][EVENT]167130 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:33.942.696 [graph_manager.cc:1130][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.942.732 [graph_manager.cc:1131][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.942.764 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.942.788 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.942.798 [graph_manager.cc:2837][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [47] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.942.934 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [25] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.942.951 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.942.964 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.942.973 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of BitcastPass is [0] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.942.997 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [8] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.943.006 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [16] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:33.943.017 [graph_manager.cc:2864][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [200] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.029 [graph_manager.cc:2872][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.049 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.068 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.089 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.106 [compile_nodes_pass.cc:88][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.119 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.131 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.247 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [103] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.299 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.316 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.331 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.345 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.354 [graph_manager.cc:2927][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [310] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.366 [graph_manager.cc:2937][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.382 [graph_manager.cc:2943][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.393 [graph_manager.cc:2950][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.592 [graph_manager.cc:2958][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [54] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.627 [graph_manager.cc:1132][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [879] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.720 [graph_manager.cc:1135][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [68] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.759 [graph_manager.cc:2975][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.791 [graph_manager.cc:2981][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.806 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.816 [graph_manager.cc:2986][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.943.825 [graph_manager.cc:1136][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [89] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.944.152 [graph_manager.cc:3555][EVENT]167130 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [285] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.944.283 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.944.313 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.944.462 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [138] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.944.497 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.944.540 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.944.566 [graph_builder.cc:865][EVENT]167130 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [346] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:33.944.953 [logger.cc:1071] 167130 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:37:33.944.987 [task_generator.cc:804][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [80] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.945.071 [task_generator.cc:805][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [71] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.946.908 [task_generator.cc:814][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1821] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.946.929 [task_generator.cc:954][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2022] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.947.002 [task_generator.cc:967][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [39] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:33.947.028 [logger.cc:1084] 167130 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:37:33.948.394 [graph_manager.cc:1152][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4535] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.948.448 [graph_manager.cc:1164][EVENT]167130 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:33.948.488 [graph_manager.cc:1271][EVENT]167130 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [165184] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.948.501 [graph_manager.cc:1272][EVENT]167130 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:37:33.948.819 [atrace_api.c:93](tid:167130) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:37:33.948.836 [atrace_api.c:95](tid:167130) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:37:33.978.348 [graph_converter.cc:838][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [11392] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.978.573 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [167] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.980.104 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [1500] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.980.510 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [371] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.980.540 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [404] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.980.812 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [253] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.980.901 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [61] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.980.972 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [50] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.981.437 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [449] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.981.660 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [193] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.981.695 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [228] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.981.767 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [57] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.981.830 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [46] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.981.893 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [47] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.982.123 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [217] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.982.320 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [178] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.982.339 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [198] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.982.401 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [53] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.982.460 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [45] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.982.478 [graph_converter.cc:849][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4079] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.983.199 [graph_converter.cc:853][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [707] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.985.192 [graph_converter.cc:857][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1949] micro second. [INFO] GE(164040,python):2024-01-10-11:37:33.985.596 [graph_converter.cc:862][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [371] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.060.112 [graph_var_manager.cc:1424][EVENT]167130 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:34.060.219 [graph_manager.cc:1248][EVENT]167130 PreRun:PreRun start: graph node size 4, session id 11, graph id 10, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:34.060.486 [atrace_api.c:28](tid:167130) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:34.060.514 [trace_rb_log.c:84](tid:167130) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:34.060.528 [atrace_api.c:32](tid:167130) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:34.060.545 [client_manager.cpp:157][SetProfilingCallback][tid:167130] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:34.060.905 [parallel_partitioner.cc:165][EVENT]167130 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.060.946 [parallel_partitioner.cc:178][EVENT]167130 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.060.995 [graph_prepare.cc:1378][EVENT]167130 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.061.149 [graph_manager.cc:1050][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [171] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.061.174 [graph_manager.cc:1052][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.061.318 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.061.350 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.061.405 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [43] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.061.419 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.061.467 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.061.481 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.061.498 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.061.604 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.061.625 [graph_manager.cc:1054][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [438] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.061.924 [graph_manager.cc:1055][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [258] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.063.114 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:34.063.143 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.063.155 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.063.164 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [363] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.063.174 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.063.183 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:34.063.191 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [76] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.063.200 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.063.208 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [8] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.065.449 [graph_manager.cc:1056][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3503] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.065.521 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.065.541 [graph_prepare.cc:1982][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [57] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.066.061 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:34.066.090 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.101 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.111 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [318] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.120 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.129 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:34.066.138 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.146 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.155 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.180 [graph_prepare.cc:1983][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [625] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.066.215 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.066.227 [graph_prepare.cc:1984][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.066.241 [graph_prepare.cc:1985][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.066.255 [graph_prepare.cc:1986][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.066.267 [graph_prepare.cc:1987][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.066.282 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.066.294 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.066.308 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.066.403 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.415 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.424 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.432 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.441 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.449 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.457 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.466 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.474 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.482 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.490 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.499 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SnapshotPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.507 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.515 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.530 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.539 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.066.562 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.066.576 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.066.610 [graph_prepare.cc:1988][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [334] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.066.623 [graph_manager.cc:1065][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1140] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.078.665 [graph_manager.cc:1077][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12021] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.078.742 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.078.799 [graph_manager.cc:1080][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [96] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.083.499 [graph_manager.cc:1081][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [4683] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.083.543 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.083.559 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.083.570 [graph_manager.cc:1082][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.083.602 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.083.618 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.083.633 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.083.668 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [25] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.083.683 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.083.697 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.083.709 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.083.752 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [33] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.083.772 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.083.803 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.083.832 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.083.848 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.083.861 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.083.870 [graph_manager.cc:2700][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [274] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.083.991 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.084.005 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.084.015 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.084.024 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.084.032 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.084.041 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CastRemovePass is [8] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.084.049 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.084.057 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.084.066 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.084.074 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.084.082 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.084.090 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.084.099 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.084.107 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.084.115 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.084.124 [graph_manager.cc:2741][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [235] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.133 [graph_manager.cc:2752][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.164 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.175 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.193 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.207 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.219 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.232 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.251 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.266 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.280 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.290 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.302 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.313 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.334 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.346 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.355 [graph_manager.cc:2810][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [195] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.385 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.084.397 [graph_manager.cc:2821][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [34] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.425 [graph_manager.cc:1087][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [837] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.560 [graph_manager.cc:1088][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [122] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.601 [graph_manager.cc:1089][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.620 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.084.635 [graph_manager.cc:1097][EVENT]167130 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:34.084.656 [graph_manager.cc:3325][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.085.739 [engine_place.cc:144][EVENT]167130 Run:The time cost of AIcoreEngine::CheckSupported is [944] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.085.769 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.085.780 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.085.859 [graph_manager.cc:3351][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [1180] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.085.877 [graph_manager.cc:3364][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.085.940 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.085.957 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.086.137 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [170] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.086.183 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.086.233 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.086.266 [graph_manager.cc:3405][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [376] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.086.284 [graph_manager.cc:3412][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.083 [graph_manager.cc:3422][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [21784] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.130 [graph_manager.cc:3428][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.285 [graph_manager.cc:3467][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [133] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.305 [graph_manager.cc:3377][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [22416] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.322 [graph_manager.cc:1106][EVENT]167130 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [23673] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.335 [graph_manager.cc:1115][EVENT]167130 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:34.108.359 [graph_manager.cc:1130][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.391 [graph_manager.cc:1131][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.430 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.448 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.458 [graph_manager.cc:2837][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.547 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.108.560 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.108.569 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.108.578 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of BitcastPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.108.587 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.108.596 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.108.605 [graph_manager.cc:2864][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [131] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.617 [graph_manager.cc:2872][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.639 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.652 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.668 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.682 [compile_nodes_pass.cc:88][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.693 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.703 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.795 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [83] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.828 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.844 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.861 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.884 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.894 [graph_manager.cc:2927][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [260] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.907 [graph_manager.cc:2937][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.922 [graph_manager.cc:2943][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.108.932 [graph_manager.cc:2950][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.109.113 [graph_manager.cc:2958][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [40] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.109.145 [graph_manager.cc:1132][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [728] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.109.220 [graph_manager.cc:1135][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [62] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.109.254 [graph_manager.cc:2975][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.109.285 [graph_manager.cc:2981][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.109.299 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.109.309 [graph_manager.cc:2986][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.109.318 [graph_manager.cc:1136][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [81] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.109.442 [graph_manager.cc:3555][EVENT]167130 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [92] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.109.537 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.109.554 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.109.734 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [170] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.109.771 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.109.814 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.109.837 [graph_builder.cc:865][EVENT]167130 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [338] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:34.110.144 [logger.cc:1071] 167130 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:37:34.110.187 [task_generator.cc:804][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [97] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.110.254 [task_generator.cc:805][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [54] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.110.979 [task_generator.cc:814][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [710] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.110.993 [task_generator.cc:954][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [904] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.111.052 [task_generator.cc:967][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [33] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:34.111.071 [logger.cc:1084] 167130 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:37:34.112.408 [graph_manager.cc:1152][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3063] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.112.448 [graph_manager.cc:1164][EVENT]167130 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:34.112.490 [graph_manager.cc:1271][EVENT]167130 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [51680] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.112.505 [graph_manager.cc:1272][EVENT]167130 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:37:34.112.832 [atrace_api.c:93](tid:167130) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:37:34.112.852 [atrace_api.c:95](tid:167130) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:37:34.136.633 [graph_converter.cc:838][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [7735] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.136.855 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [162] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.137.426 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [544] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.137.655 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [199] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.137.679 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [226] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.138.043 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [300] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.138.095 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.138.131 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.138.348 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [204] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.138.441 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [71] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.138.456 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [87] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.138.488 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.138.513 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.138.557 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.138.642 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [74] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.138.716 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [63] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.138.728 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [75] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.138.757 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.138.782 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.138.796 [graph_converter.cc:849][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [2110] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.139.041 [graph_converter.cc:853][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [235] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.139.844 [graph_converter.cc:857][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [787] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.140.007 [graph_converter.cc:862][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [134] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.212.614 [graph_var_manager.cc:1424][EVENT]167129 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:34.212.717 [graph_manager.cc:1248][EVENT]167129 PreRun:PreRun start: graph node size 4, session id 12, graph id 11, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:34.212.987 [atrace_api.c:28](tid:167129) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:34.213.014 [trace_rb_log.c:84](tid:167129) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:34.213.028 [atrace_api.c:32](tid:167129) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:34.213.046 [client_manager.cpp:157][SetProfilingCallback][tid:167129] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:34.213.399 [parallel_partitioner.cc:165][EVENT]167129 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.213.438 [parallel_partitioner.cc:178][EVENT]167129 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.213.485 [graph_prepare.cc:1378][EVENT]167129 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.213.621 [graph_manager.cc:1050][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [152] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.213.644 [graph_manager.cc:1052][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.213.826 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.213.886 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.213.937 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.213.950 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.213.996 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.214.010 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.214.027 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.214.129 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.214.151 [graph_manager.cc:1054][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [494] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.214.374 [graph_manager.cc:1055][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [210] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.215.454 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [5] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:34.215.485 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.215.497 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.215.507 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [349] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.215.515 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [15] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.215.524 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [5] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:34.215.533 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.215.541 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.215.550 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [10] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.217.752 [graph_manager.cc:1056][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3355] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.217.823 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.217.842 [graph_prepare.cc:1982][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.218.323 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:34.218.352 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [0] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.374 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.384 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [274] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.393 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.402 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:34.218.410 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.419 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.427 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.454 [graph_prepare.cc:1983][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [598] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.218.477 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.218.488 [graph_prepare.cc:1984][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.218.502 [graph_prepare.cc:1985][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.218.517 [graph_prepare.cc:1986][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.218.528 [graph_prepare.cc:1987][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.218.543 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.218.555 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.218.569 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.218.663 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.674 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.683 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.692 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.700 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.709 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.717 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.733 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.742 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of StopGradientPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.750 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.759 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.767 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.776 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.784 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.792 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.800 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.218.823 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.218.837 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.218.870 [graph_prepare.cc:1988][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [332] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.218.883 [graph_manager.cc:1065][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1095] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.231.276 [graph_manager.cc:1077][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12373] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.231.349 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.231.402 [graph_manager.cc:1080][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [88] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.239.846 [graph_manager.cc:1081][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [8428] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.239.890 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.239.906 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.239.919 [graph_manager.cc:1082][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.239.951 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.239.965 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.239.992 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.090 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [87] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.107 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.153 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.167 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.210 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.228 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.247 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.274 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.290 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.302 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.311 [graph_manager.cc:2700][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [366] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.438 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.240.451 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.240.461 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.240.469 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.240.478 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.240.487 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CastRemovePass is [9] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.240.495 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.240.503 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.240.512 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.240.520 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.240.538 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.240.547 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.240.555 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.240.564 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.240.572 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.240.581 [graph_manager.cc:2741][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [251] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.590 [graph_manager.cc:2752][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.614 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.624 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.642 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.658 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.669 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.681 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.699 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.713 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.726 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.736 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.748 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.759 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.777 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.790 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.799 [graph_manager.cc:2810][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [189] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.827 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.240.845 [graph_manager.cc:2821][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.240.874 [graph_manager.cc:1087][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [936] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.241.009 [graph_manager.cc:1088][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [122] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.241.051 [graph_manager.cc:1089][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.241.069 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.241.085 [graph_manager.cc:1097][EVENT]167129 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:34.241.106 [graph_manager.cc:3325][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.241.506 [engine_place.cc:144][EVENT]167129 Run:The time cost of AIcoreEngine::CheckSupported is [292] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.241.533 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.241.543 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.241.621 [graph_manager.cc:3351][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [502] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.241.640 [graph_manager.cc:3364][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.241.737 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.241.756 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.241.918 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [149] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.241.967 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.242.016 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.242.052 [graph_manager.cc:3405][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [400] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.242.074 [graph_manager.cc:3412][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.245 [graph_manager.cc:3422][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [18158] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.293 [graph_manager.cc:3428][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.428 [graph_manager.cc:3467][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [115] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.446 [graph_manager.cc:3377][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [18795] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.461 [graph_manager.cc:1106][EVENT]167129 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [19361] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.474 [graph_manager.cc:1115][EVENT]167129 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:34.260.496 [graph_manager.cc:1130][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.528 [graph_manager.cc:1131][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.553 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.570 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.580 [graph_manager.cc:2837][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.664 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.260.677 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.260.686 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.260.695 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.260.704 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.260.712 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [8] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.260.722 [graph_manager.cc:2864][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [125] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.733 [graph_manager.cc:2872][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.753 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.767 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.783 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.796 [compile_nodes_pass.cc:88][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.814 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.824 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.905 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [72] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.933 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.947 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.961 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.973 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.981 [graph_manager.cc:2927][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [232] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.260.993 [graph_manager.cc:2937][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.261.007 [graph_manager.cc:2943][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.261.018 [graph_manager.cc:2950][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.261.192 [graph_manager.cc:2958][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.261.223 [graph_manager.cc:1132][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [682] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.261.293 [graph_manager.cc:1135][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [57] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.261.325 [graph_manager.cc:2975][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.261.355 [graph_manager.cc:2981][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.261.369 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.261.379 [graph_manager.cc:2986][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.261.388 [graph_manager.cc:1136][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [78] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.261.527 [graph_manager.cc:3555][EVENT]167129 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [106] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.261.632 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.261.650 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.261.824 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [164] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.261.858 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.261.896 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.261.919 [graph_builder.cc:865][EVENT]167129 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [324] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:34.262.210 [logger.cc:1071] 167129 ModelBindStream: model_id=1856, stream_id=65, flag=0. [INFO] GE(164040,python):2024-01-10-11:37:34.262.239 [task_generator.cc:804][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [79] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.262.301 [task_generator.cc:805][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [50] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.263.006 [task_generator.cc:814][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [690] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.263.020 [task_generator.cc:954][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [861] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.263.077 [task_generator.cc:967][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [32] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:34.263.095 [logger.cc:1084] 167129 ModelUnbindStream: model_id=1856, stream_id=65, [INFO] GE(164040,python):2024-01-10-11:37:34.263.717 [graph_manager.cc:1152][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2302] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.263.750 [graph_manager.cc:1164][EVENT]167129 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:34.263.785 [graph_manager.cc:1271][EVENT]167129 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [50477] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.263.797 [graph_manager.cc:1272][EVENT]167129 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:37:34.264.110 [atrace_api.c:93](tid:167129) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:37:34.264.126 [atrace_api.c:95](tid:167129) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:37:34.275.921 [graph_converter.cc:838][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [3905] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.276.091 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [120] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.276.604 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [486] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.276.822 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [191] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.276.845 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [217] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.277.082 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [212] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.277.122 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.277.154 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.277.355 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [190] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.277.441 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [67] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.277.456 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [82] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.277.485 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.277.510 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.277.537 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.277.616 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [70] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.277.731 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [57] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.277.745 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [117] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.277.772 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.277.796 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.277.809 [graph_converter.cc:849][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1844] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.278.034 [graph_converter.cc:853][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [215] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.278.774 [graph_converter.cc:857][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [724] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.278.927 [graph_converter.cc:862][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [127] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.357.654 [graph_var_manager.cc:1424][EVENT]167130 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:34.357.819 [graph_manager.cc:1248][EVENT]167130 PreRun:PreRun start: graph node size 4, session id 13, graph id 12, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:34.358.087 [atrace_api.c:28](tid:167130) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:34.358.116 [trace_rb_log.c:84](tid:167130) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:34.358.129 [atrace_api.c:32](tid:167130) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:34.358.147 [client_manager.cpp:157][SetProfilingCallback][tid:167130] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:34.358.527 [parallel_partitioner.cc:165][EVENT]167130 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.358.568 [parallel_partitioner.cc:178][EVENT]167130 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.358.617 [graph_prepare.cc:1378][EVENT]167130 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.358.757 [graph_manager.cc:1050][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [156] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.358.786 [graph_manager.cc:1052][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.358.940 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.358.972 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.359.021 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.359.034 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.359.078 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.359.093 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.359.110 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.359.213 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.359.233 [graph_manager.cc:1054][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [431] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.359.455 [graph_manager.cc:1055][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [207] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.360.594 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:34.360.626 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.360.638 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.360.648 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [363] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.360.657 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.360.666 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:34.360.674 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.360.694 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.360.703 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.362.925 [graph_manager.cc:1056][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3445] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.362.997 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.016 [graph_prepare.cc:1982][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.363.445 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:34.363.473 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.485 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.494 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [239] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.503 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.512 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:34.363.521 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.529 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.537 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.562 [graph_prepare.cc:1983][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [532] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.363.588 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.363.599 [graph_prepare.cc:1984][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.363.613 [graph_prepare.cc:1985][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.363.628 [graph_prepare.cc:1986][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.363.640 [graph_prepare.cc:1987][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.363.654 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.363.666 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.363.690 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.363.783 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.795 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.804 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.813 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.821 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.830 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.838 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [0] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.846 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.855 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.863 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.871 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.879 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SnapshotPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.888 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.896 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [8] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.904 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.912 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.363.934 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.363.947 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.363.980 [graph_prepare.cc:1988][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [331] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.363.993 [graph_manager.cc:1065][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1030] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.376.816 [graph_manager.cc:1077][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12802] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.376.882 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.376.958 [graph_manager.cc:1080][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [103] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.380.457 [graph_manager.cc:1081][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3481] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.380.503 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.380.521 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.380.533 [graph_manager.cc:1082][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.380.568 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.380.585 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.380.599 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.380.706 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [97] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.380.724 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.380.779 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [42] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.380.794 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.380.836 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [30] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.380.854 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.380.883 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.380.911 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.380.926 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.380.939 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.380.948 [graph_manager.cc:2700][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [385] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.071 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.381.085 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.381.095 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.381.115 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.381.125 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.381.133 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.381.142 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.381.150 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.381.158 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.381.167 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.381.175 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.381.183 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.381.191 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.381.200 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.381.208 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.381.217 [graph_manager.cc:2741][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [251] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.226 [graph_manager.cc:2752][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.250 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.262 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.280 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.294 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.306 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.318 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.339 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.353 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.374 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.384 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.397 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.408 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.427 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.440 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.449 [graph_manager.cc:2810][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [203] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.478 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.381.489 [graph_manager.cc:2821][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.517 [graph_manager.cc:1087][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [961] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.650 [graph_manager.cc:1088][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [119] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.721 [graph_manager.cc:1089][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [51] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.742 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.381.756 [graph_manager.cc:1097][EVENT]167130 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:34.381.776 [graph_manager.cc:3325][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.382.154 [engine_place.cc:144][EVENT]167130 Run:The time cost of AIcoreEngine::CheckSupported is [278] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.382.182 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.382.192 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.382.277 [graph_manager.cc:3351][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [488] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.382.297 [graph_manager.cc:3364][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.382.365 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.382.382 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.382.542 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [141] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.382.588 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.382.633 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.382.668 [graph_manager.cc:3405][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [357] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.382.685 [graph_manager.cc:3412][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.005 [graph_manager.cc:3422][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [101305] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.075 [graph_manager.cc:3428][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.264 [graph_manager.cc:3467][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [165] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.283 [graph_manager.cc:3377][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [101974] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.301 [graph_manager.cc:1106][EVENT]167130 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [102531] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.312 [graph_manager.cc:1115][EVENT]167130 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:34.484.338 [graph_manager.cc:1130][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.374 [graph_manager.cc:1131][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.401 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.419 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.429 [graph_manager.cc:2837][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.531 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [21] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.484.544 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.484.553 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.484.562 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.484.571 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.484.597 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [7] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.484.608 [graph_manager.cc:2864][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [162] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.620 [graph_manager.cc:2872][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.642 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.656 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.672 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.687 [compile_nodes_pass.cc:88][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.697 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.706 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.791 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [75] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.836 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.850 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.863 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.877 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.887 [graph_manager.cc:2927][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [249] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.899 [graph_manager.cc:2937][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.914 [graph_manager.cc:2943][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.484.925 [graph_manager.cc:2950][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.485.113 [graph_manager.cc:2958][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.485.147 [graph_manager.cc:1132][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [759] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.485.237 [graph_manager.cc:1135][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [77] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.485.283 [graph_manager.cc:2975][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.485.315 [graph_manager.cc:2981][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.485.329 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.485.340 [graph_manager.cc:2986][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.485.349 [graph_manager.cc:1136][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [85] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.485.490 [graph_manager.cc:3555][EVENT]167130 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [107] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.485.589 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.485.607 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.485.805 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [188] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.485.842 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [24] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.485.886 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [33] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.485.910 [graph_builder.cc:865][EVENT]167130 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [360] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:34.486.270 [logger.cc:1071] 167130 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:37:34.486.302 [task_generator.cc:804][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [92] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.486.365 [task_generator.cc:805][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [51] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.487.243 [task_generator.cc:814][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [862] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.487.258 [task_generator.cc:954][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1048] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.487.324 [task_generator.cc:967][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [38] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:34.487.343 [logger.cc:1084] 167130 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:37:34.489.137 [graph_manager.cc:1152][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3760] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.489.185 [graph_manager.cc:1164][EVENT]167130 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:34.489.242 [graph_manager.cc:1271][EVENT]167130 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [130804] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.489.255 [graph_manager.cc:1272][EVENT]167130 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:37:34.489.584 [atrace_api.c:93](tid:167130) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:37:34.489.605 [atrace_api.c:95](tid:167130) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:37:34.521.682 [graph_converter.cc:838][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [10264] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.521.898 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [140] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.522.670 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [743] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.522.923 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [223] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.522.948 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [250] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.523.149 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [188] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.523.202 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.523.243 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [26] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.523.510 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [255] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.523.624 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [92] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.523.641 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [109] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.523.677 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.523.709 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.523.744 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.523.860 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [107] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.523.955 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [81] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.523.968 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [96] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.524.002 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [25] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.524.034 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.524.049 [graph_converter.cc:849][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [2297] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.524.390 [graph_converter.cc:853][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [330] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.525.409 [graph_converter.cc:857][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [986] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.525.620 [graph_converter.cc:862][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [181] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.601.820 [graph_var_manager.cc:1424][EVENT]167129 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:34.601.923 [graph_manager.cc:1248][EVENT]167129 PreRun:PreRun start: graph node size 4, session id 14, graph id 13, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:34.602.159 [atrace_api.c:28](tid:167129) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:34.602.190 [trace_rb_log.c:84](tid:167129) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:34.602.203 [atrace_api.c:32](tid:167129) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:34.602.221 [client_manager.cpp:157][SetProfilingCallback][tid:167129] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:34.602.599 [parallel_partitioner.cc:165][EVENT]167129 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.602.635 [parallel_partitioner.cc:178][EVENT]167129 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.602.682 [graph_prepare.cc:1378][EVENT]167129 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.602.816 [graph_manager.cc:1050][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [151] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.602.840 [graph_manager.cc:1052][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.602.979 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.603.008 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.603.056 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.603.069 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.603.116 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.603.130 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.603.147 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.603.248 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.603.268 [graph_manager.cc:1054][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [415] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.603.516 [graph_manager.cc:1055][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [207] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.604.656 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:34.604.685 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.604.697 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.604.706 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [445] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.604.715 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.604.724 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:34.604.732 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.604.741 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.604.749 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.607.857 [graph_manager.cc:1056][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [4322] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.607.926 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.607.946 [graph_prepare.cc:1982][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.608.488 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:34.608.516 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.527 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.537 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [341] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.546 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.554 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:34.608.563 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.571 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.579 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.605 [graph_prepare.cc:1983][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [645] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.608.641 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.608.653 [graph_prepare.cc:1984][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.608.667 [graph_prepare.cc:1985][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.608.682 [graph_prepare.cc:1986][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.608.693 [graph_prepare.cc:1987][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.608.707 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.608.718 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.608.733 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.608.825 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.836 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.846 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.854 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.863 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.871 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.880 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.888 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.897 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.905 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.913 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.922 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.930 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.938 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.947 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.970 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.608.993 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.609.007 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.609.039 [graph_prepare.cc:1988][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [338] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.609.053 [graph_manager.cc:1065][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1160] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.622.358 [graph_manager.cc:1077][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13285] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.622.433 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.622.483 [graph_manager.cc:1080][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [85] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.633.659 [graph_manager.cc:1081][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [11156] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.633.734 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.633.752 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.633.765 [graph_manager.cc:1082][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [40] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.633.798 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.633.815 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.633.829 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.633.981 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [141] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.633.999 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.090 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [80] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.106 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.157 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [40] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.178 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.213 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.303 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [79] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.320 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.334 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.344 [graph_manager.cc:2700][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [552] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.582 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [4] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.634.598 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.634.608 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.634.617 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.634.625 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.634.634 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CastRemovePass is [40] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.634.643 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.634.651 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [7] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.634.659 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [9] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.634.667 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [4] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.634.676 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [15] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.634.684 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.634.692 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.634.701 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [8] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.634.709 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [4] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.634.719 [graph_manager.cc:2741][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [355] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.728 [graph_manager.cc:2752][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.753 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.775 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.797 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.814 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.826 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.839 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.859 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.874 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.887 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.897 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.909 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.921 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.943 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.956 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.634.965 [graph_manager.cc:2810][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [218] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.635.009 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.635.021 [graph_manager.cc:2821][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [47] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.635.049 [graph_manager.cc:1087][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1265] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.635.605 [graph_manager.cc:1088][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [543] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.635.669 [graph_manager.cc:1089][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.635.692 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.635.709 [graph_manager.cc:1097][EVENT]167129 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:34.635.732 [graph_manager.cc:3325][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.646.406 [engine_place.cc:144][EVENT]167129 Run:The time cost of AIcoreEngine::CheckSupported is [10447] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.646.440 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.646.451 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.646.544 [graph_manager.cc:3351][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10789] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.646.563 [graph_manager.cc:3364][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.646.643 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.646.674 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.646.844 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [158] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.646.886 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [29] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.646.936 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.646.972 [graph_manager.cc:3405][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [396] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.646.991 [graph_manager.cc:3412][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.062 [graph_manager.cc:3422][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [117058] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.113 [graph_manager.cc:3428][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.286 [graph_manager.cc:3467][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [152] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.306 [graph_manager.cc:3377][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [117730] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.325 [graph_manager.cc:1106][EVENT]167129 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [128601] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.338 [graph_manager.cc:1115][EVENT]167129 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:34.764.362 [graph_manager.cc:1130][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.396 [graph_manager.cc:1131][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.449 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.470 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.484 [graph_manager.cc:2837][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [50] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.624 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [26] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.764.641 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.764.650 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.764.663 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.764.672 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [9] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.764.680 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [16] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.764.690 [graph_manager.cc:2864][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [187] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.703 [graph_manager.cc:2872][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.724 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.740 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.757 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.773 [compile_nodes_pass.cc:88][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.784 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.794 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.904 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [100] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.959 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.977 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.764.995 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.765.010 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.765.029 [graph_manager.cc:2927][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [308] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.765.042 [graph_manager.cc:2937][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.765.057 [graph_manager.cc:2943][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.765.069 [graph_manager.cc:2950][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.765.266 [graph_manager.cc:2958][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [56] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.765.299 [graph_manager.cc:1132][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [866] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.765.379 [graph_manager.cc:1135][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [67] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.765.417 [graph_manager.cc:2975][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.765.448 [graph_manager.cc:2981][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.765.463 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.765.473 [graph_manager.cc:2986][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.765.482 [graph_manager.cc:1136][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [87] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.765.881 [graph_manager.cc:3555][EVENT]167129 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [360] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.766.017 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [33] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.766.048 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.766.193 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [134] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.766.229 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [24] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.766.271 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.766.297 [graph_builder.cc:865][EVENT]167129 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [344] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:34.766.687 [logger.cc:1071] 167129 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:37:34.766.730 [task_generator.cc:804][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [98] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.766.816 [task_generator.cc:805][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [73] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.768.619 [task_generator.cc:814][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1787] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.768.637 [task_generator.cc:954][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2006] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.768.709 [task_generator.cc:967][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [38] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:34.768.738 [logger.cc:1084] 167129 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:37:34.770.139 [graph_manager.cc:1152][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4624] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.770.183 [graph_manager.cc:1164][EVENT]167129 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:34.770.225 [graph_manager.cc:1271][EVENT]167129 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [167713] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.770.240 [graph_manager.cc:1272][EVENT]167129 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:37:34.770.561 [atrace_api.c:93](tid:167129) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:37:34.770.580 [atrace_api.c:95](tid:167129) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:37:34.801.174 [graph_converter.cc:838][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [11141] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.801.400 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [165] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.802.970 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [1540] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.803.366 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [361] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.803.394 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [393] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.803.662 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [254] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.803.746 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [60] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.803.814 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [49] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.804.277 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [446] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.804.492 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [186] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.804.518 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [214] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.804.582 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [52] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.804.640 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [44] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.804.716 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [48] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.804.942 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [213] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.805.134 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [172] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.805.151 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [191] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.805.212 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [51] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.805.268 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [44] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.805.287 [graph_converter.cc:849][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4060] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.806.016 [graph_converter.cc:853][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [718] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.807.972 [graph_converter.cc:857][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1928] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.808.372 [graph_converter.cc:862][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [369] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.882.617 [graph_var_manager.cc:1424][EVENT]167130 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:34.882.718 [graph_manager.cc:1248][EVENT]167130 PreRun:PreRun start: graph node size 4, session id 15, graph id 14, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:34.882.956 [atrace_api.c:28](tid:167130) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:34.882.990 [trace_rb_log.c:84](tid:167130) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:34.883.004 [atrace_api.c:32](tid:167130) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:34.883.021 [client_manager.cpp:157][SetProfilingCallback][tid:167130] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:34.883.381 [parallel_partitioner.cc:165][EVENT]167130 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.883.421 [parallel_partitioner.cc:178][EVENT]167130 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.883.468 [graph_prepare.cc:1378][EVENT]167130 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.883.630 [graph_manager.cc:1050][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [180] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.883.658 [graph_manager.cc:1052][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.883.797 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.883.829 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.883.908 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.883.921 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.883.964 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.883.978 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.883.994 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.884.095 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.884.116 [graph_manager.cc:1054][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [446] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.884.338 [graph_manager.cc:1055][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [208] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.885.487 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:34.885.519 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.885.531 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.885.544 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [446] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.885.553 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.885.566 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:34.885.575 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [10] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.885.584 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.885.592 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.888.512 [graph_manager.cc:1056][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [4152] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.888.584 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.888.604 [graph_prepare.cc:1982][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [55] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.889.151 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:34.889.180 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.203 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.214 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [344] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.223 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.232 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [4] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:34.889.240 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.249 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.257 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.283 [graph_prepare.cc:1983][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [665] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.889.307 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.889.318 [graph_prepare.cc:1984][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.889.332 [graph_prepare.cc:1985][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.889.346 [graph_prepare.cc:1986][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.889.357 [graph_prepare.cc:1987][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.889.371 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.889.383 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.889.397 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.889.489 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.501 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.510 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.519 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.527 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.535 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.544 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.559 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.568 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.577 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.585 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.594 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.602 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.611 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.619 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.627 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:34.889.651 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.889.663 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.889.745 [graph_prepare.cc:1988][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [378] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.889.759 [graph_manager.cc:1065][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1210] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.902.761 [graph_manager.cc:1077][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12982] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.902.836 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.902.888 [graph_manager.cc:1080][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [87] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.913.951 [graph_manager.cc:1081][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [11044] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.913.999 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.914.016 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.914.028 [graph_manager.cc:1082][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.914.062 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.914.078 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.914.108 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.914.261 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [141] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.914.279 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.914.373 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [83] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.914.389 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.914.440 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [40] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.914.462 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.914.482 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.914.574 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [80] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.914.591 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.914.604 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.914.614 [graph_manager.cc:2700][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [558] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.914.853 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.914.869 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.914.879 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.914.888 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.914.897 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.914.906 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CastRemovePass is [38] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.914.914 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [5] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.914.923 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.914.931 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [10] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.914.939 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.914.958 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.914.967 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.914.975 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.914.983 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [9] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.914.992 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [4] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.915.001 [graph_manager.cc:2741][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [368] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.011 [graph_manager.cc:2752][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.036 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.049 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.071 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.089 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.101 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.114 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.135 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.149 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.162 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.173 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.185 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.196 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.219 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.233 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.242 [graph_manager.cc:2810][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [211] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.286 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:34.915.307 [graph_manager.cc:2821][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [57] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.337 [graph_manager.cc:1087][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1289] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.893 [graph_manager.cc:1088][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [540] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.957 [graph_manager.cc:1089][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.980 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.915.998 [graph_manager.cc:1097][EVENT]167130 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:34.916.020 [graph_manager.cc:3325][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.926.584 [engine_place.cc:144][EVENT]167130 Run:The time cost of AIcoreEngine::CheckSupported is [10344] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.926.618 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.926.629 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.926.724 [graph_manager.cc:3351][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10690] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.926.745 [graph_manager.cc:3364][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.926.823 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.926.854 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.927.025 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [160] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.927.067 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.927.117 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.927.153 [graph_manager.cc:3405][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [395] micro second. [INFO] GE(164040,python):2024-01-10-11:37:34.927.171 [graph_manager.cc:3412][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.045.412 [graph_manager.cc:3422][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [118225] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.045.468 [graph_manager.cc:3428][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.045.660 [graph_manager.cc:3467][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [154] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.045.681 [graph_manager.cc:3377][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [118924] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.045.739 [graph_manager.cc:1106][EVENT]167130 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [129726] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.045.752 [graph_manager.cc:1115][EVENT]167130 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:35.045.779 [graph_manager.cc:1130][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.045.813 [graph_manager.cc:1131][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.045.843 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.045.863 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.045.873 [graph_manager.cc:2837][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [44] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.012 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [25] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:35.046.026 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:35.046.036 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:35.046.045 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:35.046.053 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [9] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:35.046.062 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [15] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:35.046.072 [graph_manager.cc:2864][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [181] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.085 [graph_manager.cc:2872][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.105 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.122 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.139 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.153 [compile_nodes_pass.cc:88][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.173 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [24] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.184 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.294 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [101] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.345 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.359 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.374 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.389 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.398 [graph_manager.cc:2927][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [297] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.410 [graph_manager.cc:2937][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.425 [graph_manager.cc:2943][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.436 [graph_manager.cc:2950][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.636 [graph_manager.cc:2958][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [57] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.671 [graph_manager.cc:1132][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [844] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.752 [graph_manager.cc:1135][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [68] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.791 [graph_manager.cc:2975][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.823 [graph_manager.cc:2981][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.837 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.848 [graph_manager.cc:2986][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.046.857 [graph_manager.cc:1136][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [88] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.047.188 [graph_manager.cc:3555][EVENT]167130 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [290] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.047.332 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [33] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.047.362 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.047.510 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [137] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.047.546 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.047.591 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [33] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.047.617 [graph_builder.cc:865][EVENT]167130 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [350] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:35.048.008 [logger.cc:1071] 167130 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:37:35.048.042 [task_generator.cc:804][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [81] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.048.124 [task_generator.cc:805][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [70] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.049.942 [task_generator.cc:814][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1802] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.049.959 [task_generator.cc:954][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1998] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.050.032 [task_generator.cc:967][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [39] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:35.050.057 [logger.cc:1084] 167130 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:37:35.051.459 [graph_manager.cc:1152][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4567] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.051.498 [graph_manager.cc:1164][EVENT]167130 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:35.051.538 [graph_manager.cc:1271][EVENT]167130 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [168246] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.051.551 [graph_manager.cc:1272][EVENT]167130 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:37:35.051.876 [atrace_api.c:93](tid:167130) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:37:35.051.893 [atrace_api.c:95](tid:167130) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:37:35.082.233 [graph_converter.cc:838][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [11237] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.082.459 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [165] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.084.011 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [1521] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.084.412 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [367] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.084.440 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [399] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.084.727 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [256] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.084.811 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [59] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.084.879 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [49] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.085.345 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [449] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.085.564 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [190] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.085.587 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [215] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.085.654 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [55] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.085.774 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [105] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.085.840 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [48] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.086.067 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [213] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.086.260 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [176] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.086.278 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [196] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.086.339 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [52] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.086.397 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [46] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.086.417 [graph_converter.cc:849][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4131] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.087.127 [graph_converter.cc:853][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [699] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.089.098 [graph_converter.cc:857][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1944] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.089.500 [graph_converter.cc:862][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [370] micro second. [INFO] HCCP(164040,python):2024-01-10-11:37:35.107.523 [ra_host.c:1761]tid:168537,ra_socket_white_list_add(1761) : Input parameters: phy_id[1], local_ip[1.0.0.0], num[1] [INFO] HCCP(164040,python):2024-01-10-11:37:35.107.781 [ra_host.c:825]tid:168537,ra_socket_batch_connect(825) : Input parameters: [0]th, phy_id[1], local_ip[1.0.0.0], remote_ip[0.0.0.0], tag:[8.92.9.85%enp189s0f0_60000_0_1704857845691656] [INFO] HCCP(164040,python):2024-01-10-11:37:35.107.799 [ra_host.c:825]tid:168537,ra_socket_batch_connect(825) : Input parameters: [1]th, phy_id[1], local_ip[1.0.0.0], remote_ip[3.0.0.0], tag:[8.92.9.85%enp189s0f0_60000_0_1704857845691656] [INFO] HCCP(164040,python):2024-01-10-11:37:35.509.269 [ra_host.c:825]tid:167130,ra_socket_batch_connect(825) : Input parameters: [0]th, phy_id[1], local_ip[1.0.0.0], remote_ip[0.0.0.0], tag:[HeartBeat_8.92.9.85/1_to_8.92.9.85/0] [INFO] HCCP(164040,python):2024-01-10-11:37:35.509.433 [ra_host.c:1761]tid:167130,ra_socket_white_list_add(1761) : Input parameters: phy_id[1], local_ip[1.0.0.0], num[1] [INFO] HCCL(164040,python):2024-01-10-11:37:35.516.475 [hccl_impl.cc:3258][167130]resource creation success, take time [410032]us, tag[AllReduce_8.92.9.85%enp189s0f0_60000_0_1704857845691656] [INFO] GE(164040,python):2024-01-10-11:37:35.582.881 [graph_var_manager.cc:1424][EVENT]167130 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:35.582.984 [graph_manager.cc:1248][EVENT]167130 PreRun:PreRun start: graph node size 3, session id 16, graph id 15, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:35.583.263 [atrace_api.c:28](tid:167130) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:35.583.290 [trace_rb_log.c:84](tid:167130) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:35.583.303 [atrace_api.c:32](tid:167130) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:35.583.320 [client_manager.cpp:157][SetProfilingCallback][tid:167130] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:35.583.688 [parallel_partitioner.cc:165][EVENT]167130 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.583.724 [parallel_partitioner.cc:178][EVENT]167130 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.583.769 [graph_prepare.cc:1378][EVENT]167130 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.583.898 [graph_manager.cc:1050][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [144] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.583.921 [graph_manager.cc:1052][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.584.042 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.584.071 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.584.119 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.584.131 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.584.175 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.584.189 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.584.205 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.584.298 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.584.319 [graph_manager.cc:1054][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [386] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.584.546 [graph_manager.cc:1055][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [212] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.585.452 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:35.585.484 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.585.498 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.585.508 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [288] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.585.517 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.585.526 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [4] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:35.585.534 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [9] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.585.542 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.585.551 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.587.505 [graph_manager.cc:1056][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2907] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.587.572 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.587.591 [graph_prepare.cc:1982][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [49] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.587.961 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:35.587.986 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.587.996 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.005 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [169] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.014 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.022 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [6] [INFO] GE(164040,python):2024-01-10-11:37:35.588.031 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.040 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.048 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.096 [graph_prepare.cc:1983][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [492] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.588.120 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.588.143 [graph_prepare.cc:1984][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.588.158 [graph_prepare.cc:1985][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.588.172 [graph_prepare.cc:1986][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.588.183 [graph_prepare.cc:1987][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.588.198 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.588.210 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.588.224 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.588.305 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.317 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.326 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.335 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.343 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.352 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.360 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.368 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.376 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.385 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.393 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.401 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.409 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.418 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.426 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.435 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.588.463 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.588.476 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.588.506 [graph_prepare.cc:1988][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [313] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.588.519 [graph_manager.cc:1065][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [978] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.600.521 [graph_manager.cc:1077][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11983] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.600.622 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.600.676 [graph_manager.cc:1080][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [117] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.059 [graph_manager.cc:1081][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3363] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.104 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.119 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.130 [graph_manager.cc:1082][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [34] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.162 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.176 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.190 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.217 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.230 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.243 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.256 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.291 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.313 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.334 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.377 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.396 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.408 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.417 [graph_manager.cc:2700][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [261] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.521 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.604.539 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.604.552 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.604.564 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.604.576 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.604.585 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.604.593 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.604.601 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.604.610 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.604.618 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.604.626 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.604.634 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.604.642 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.604.651 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.604.662 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.604.676 [graph_manager.cc:2741][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [241] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.689 [graph_manager.cc:2752][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.716 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.731 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.763 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.783 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.798 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.814 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.835 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.853 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.872 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.887 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.901 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.913 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.934 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.950 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.959 [graph_manager.cc:2810][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [246] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.604.985 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.604.997 [graph_manager.cc:2821][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [29] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.605.025 [graph_manager.cc:1087][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [876] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.605.156 [graph_manager.cc:1088][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [118] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.605.192 [graph_manager.cc:1089][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.605.210 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.605.224 [graph_manager.cc:1097][EVENT]167130 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:35.605.244 [graph_manager.cc:3325][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.605.595 [engine_place.cc:144][EVENT]167130 Run:The time cost of AIcoreEngine::CheckSupported is [256] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.605.634 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.605.644 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.605.745 [graph_manager.cc:3351][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [488] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.605.764 [graph_manager.cc:3364][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.605.825 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.605.842 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.605.971 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [120] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.606.012 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.606.057 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [33] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.606.090 [graph_manager.cc:3405][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [314] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.606.107 [graph_manager.cc:3412][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.698.849 [graph_manager.cc:3422][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [92725] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.698.904 [graph_manager.cc:3428][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.051 [graph_manager.cc:3467][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [124] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.070 [graph_manager.cc:3377][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [93294] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.087 [graph_manager.cc:1106][EVENT]167130 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [93850] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.100 [graph_manager.cc:1115][EVENT]167130 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:35.699.123 [graph_manager.cc:1130][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.154 [graph_manager.cc:1131][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.180 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.213 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.223 [graph_manager.cc:2837][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [53] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.305 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.699.318 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.699.327 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.699.336 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.699.344 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [4] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.699.353 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [3] [INFO] GE(164040,python):2024-01-10-11:37:35.699.363 [graph_manager.cc:2864][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [122] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.375 [graph_manager.cc:2872][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.395 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.409 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.425 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.440 [compile_nodes_pass.cc:88][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.450 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.460 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.533 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [63] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.572 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.586 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.599 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.613 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.630 [graph_manager.cc:2927][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [238] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.642 [graph_manager.cc:2937][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.656 [graph_manager.cc:2943][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.667 [graph_manager.cc:2950][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.843 [graph_manager.cc:2958][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [34] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.875 [graph_manager.cc:1132][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [707] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.950 [graph_manager.cc:1135][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [62] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.699.983 [graph_manager.cc:2975][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.700.015 [graph_manager.cc:2981][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.700.029 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.700.038 [graph_manager.cc:2986][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.700.047 [graph_manager.cc:1136][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [80] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.700.173 [graph_manager.cc:3555][EVENT]167130 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [96] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.700.262 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.700.278 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.700.377 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [89] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.700.410 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.700.450 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [30] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.700.473 [graph_builder.cc:865][EVENT]167130 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [245] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:35.700.760 [logger.cc:1071] 167130 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:37:35.700.789 [task_generator.cc:804][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [84] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.700.857 [task_generator.cc:805][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [45] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.701.799 [task_generator.cc:814][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [927] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.701.816 [task_generator.cc:954][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1111] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.701.882 [task_generator.cc:967][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [37] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:35.701.902 [logger.cc:1084] 167130 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:37:35.703.644 [graph_manager.cc:1152][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3573] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.703.689 [graph_manager.cc:1164][EVENT]167130 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:35.703.726 [graph_manager.cc:1271][EVENT]167130 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [120120] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.703.741 [graph_manager.cc:1272][EVENT]167130 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:37:35.704.060 [atrace_api.c:93](tid:167130) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:37:35.704.081 [atrace_api.c:95](tid:167130) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:37:35.726.064 [graph_converter.cc:838][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [8193] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.726.253 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [133] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.726.893 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [613] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.727.114 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [192] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.727.137 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [217] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.727.324 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [175] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.727.370 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [26] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.727.404 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.727.633 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [217] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.727.734 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [80] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.727.749 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [96] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.727.782 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.727.810 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.727.840 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.727.949 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [85] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.728.030 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [69] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.728.041 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [80] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.728.071 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.728.097 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.728.110 [graph_converter.cc:849][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1997] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.728.393 [graph_converter.cc:853][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [273] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.729.252 [graph_converter.cc:857][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [843] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.729.422 [graph_converter.cc:862][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [143] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.805.974 [graph_var_manager.cc:1424][EVENT]167130 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:35.806.077 [graph_manager.cc:1248][EVENT]167130 PreRun:PreRun start: graph node size 4, session id 17, graph id 16, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:35.806.317 [atrace_api.c:28](tid:167130) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:35.806.347 [trace_rb_log.c:84](tid:167130) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:35.806.360 [atrace_api.c:32](tid:167130) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:35.806.376 [client_manager.cpp:157][SetProfilingCallback][tid:167130] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:35.806.786 [parallel_partitioner.cc:165][EVENT]167130 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.806.824 [parallel_partitioner.cc:178][EVENT]167130 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.806.870 [graph_prepare.cc:1378][EVENT]167130 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.807.492 [graph_manager.cc:1050][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [638] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.807.521 [graph_manager.cc:1052][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.807.666 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.807.698 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.807.746 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.807.782 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.807.827 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.807.841 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.807.857 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.807.955 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.807.976 [graph_manager.cc:1054][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [441] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.808.203 [graph_manager.cc:1055][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [213] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.809.278 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:35.809.308 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.809.320 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.809.330 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [341] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.809.339 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.809.348 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:35.809.356 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.809.365 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.809.373 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [9] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.811.472 [graph_manager.cc:1056][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3249] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.811.542 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.811.560 [graph_prepare.cc:1982][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.812.022 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:35.812.049 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.060 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.080 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [264] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.090 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.099 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:35.812.108 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [9] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.116 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.125 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.150 [graph_prepare.cc:1983][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [576] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.812.174 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.812.185 [graph_prepare.cc:1984][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.812.199 [graph_prepare.cc:1985][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.812.213 [graph_prepare.cc:1986][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.812.225 [graph_prepare.cc:1987][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.812.240 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.812.251 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.812.265 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.812.357 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.371 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.380 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.389 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.397 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.406 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.414 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.422 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.439 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.448 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.456 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.465 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.473 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.481 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.489 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.498 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.812.522 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.812.536 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.812.570 [graph_prepare.cc:1988][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [335] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.812.583 [graph_manager.cc:1065][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1077] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.824.906 [graph_manager.cc:1077][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12301] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.824.977 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.825.032 [graph_manager.cc:1080][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [86] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.833.656 [graph_manager.cc:1081][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [8604] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.833.735 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.833.753 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.833.765 [graph_manager.cc:1082][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.833.797 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.833.813 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.833.828 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.833.943 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [85] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.833.959 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.004 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [34] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.018 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.059 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [30] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.079 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.106 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.134 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.149 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.162 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.172 [graph_manager.cc:2700][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [380] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.303 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.834.321 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.834.330 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.834.343 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.834.355 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.834.370 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CastRemovePass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.834.379 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.834.391 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.834.402 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.834.410 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.834.422 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.834.443 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.834.455 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.834.464 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.834.472 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.834.482 [graph_manager.cc:2741][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [288] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.491 [graph_manager.cc:2752][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.514 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.526 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.544 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.562 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.578 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.594 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.617 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.635 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.651 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.664 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.679 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.694 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.718 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.734 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.745 [graph_manager.cc:2810][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [235] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.777 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.834.792 [graph_manager.cc:2821][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.833 [graph_manager.cc:1087][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1048] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.834.971 [graph_manager.cc:1088][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [121] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.835.011 [graph_manager.cc:1089][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.835.031 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.835.046 [graph_manager.cc:1097][EVENT]167130 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:35.835.066 [graph_manager.cc:3325][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.835.451 [engine_place.cc:144][EVENT]167130 Run:The time cost of AIcoreEngine::CheckSupported is [289] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.835.480 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.835.490 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.835.572 [graph_manager.cc:3351][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [494] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.835.592 [graph_manager.cc:3364][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.835.659 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.835.677 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.835.863 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [176] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.835.910 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.835.966 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [43] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.836.006 [graph_manager.cc:3405][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [402] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.836.024 [graph_manager.cc:3412][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.852.309 [graph_manager.cc:3422][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [16271] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.852.364 [graph_manager.cc:3428][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.852.531 [graph_manager.cc:3467][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [130] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.852.552 [graph_manager.cc:3377][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [16949] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.852.570 [graph_manager.cc:1106][EVENT]167130 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [17510] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.852.582 [graph_manager.cc:1115][EVENT]167130 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:35.852.607 [graph_manager.cc:1130][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.852.643 [graph_manager.cc:1131][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.852.669 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.852.686 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.852.697 [graph_manager.cc:2837][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.852.784 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.852.798 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.852.807 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.852.816 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.852.825 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.852.834 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [9] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.852.843 [graph_manager.cc:2864][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [131] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.852.856 [graph_manager.cc:2872][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.852.876 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.852.890 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.852.906 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.852.920 [compile_nodes_pass.cc:88][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.852.930 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.852.949 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.853.035 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [76] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.853.066 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.853.079 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.853.094 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.853.107 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.853.116 [graph_manager.cc:2927][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [244] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.853.129 [graph_manager.cc:2937][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.853.142 [graph_manager.cc:2943][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.853.154 [graph_manager.cc:2950][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.853.335 [graph_manager.cc:2958][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [42] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.853.371 [graph_manager.cc:1132][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [713] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.853.448 [graph_manager.cc:1135][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [64] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.853.484 [graph_manager.cc:2975][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.853.520 [graph_manager.cc:2981][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.853.534 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.853.548 [graph_manager.cc:2986][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.853.561 [graph_manager.cc:1136][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [94] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.853.749 [graph_manager.cc:3555][EVENT]167130 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [155] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.853.852 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.853.884 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.854.039 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [142] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.854.076 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.854.124 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [34] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.854.154 [graph_builder.cc:865][EVENT]167130 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [340] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:35.854.632 [logger.cc:1071] 167130 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:37:35.854.672 [task_generator.cc:804][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [187] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.854.748 [task_generator.cc:805][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [62] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.855.490 [task_generator.cc:814][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [725] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.855.508 [task_generator.cc:954][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1024] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.855.572 [task_generator.cc:967][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [36] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:35.855.591 [logger.cc:1084] 167130 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:37:35.856.919 [graph_manager.cc:1152][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3332] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.856.955 [graph_manager.cc:1164][EVENT]167130 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:35.856.997 [graph_manager.cc:1271][EVENT]167130 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [50302] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.857.009 [graph_manager.cc:1272][EVENT]167130 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:37:35.857.333 [atrace_api.c:93](tid:167130) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:37:35.857.352 [atrace_api.c:95](tid:167130) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:37:35.868.902 [graph_converter.cc:838][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [3704] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.869.077 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [125] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.869.576 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [474] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.869.809 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [207] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.869.834 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [232] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.870.059 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [213] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.870.114 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.870.147 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.870.352 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [193] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.870.439 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [69] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.870.454 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [83] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.870.484 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.870.511 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.870.539 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.870.620 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [71] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.870.688 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [57] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.870.700 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [69] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.870.726 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.870.750 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.870.764 [graph_converter.cc:849][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1819] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.870.996 [graph_converter.cc:853][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [222] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.871.739 [graph_converter.cc:857][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [722] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.871.896 [graph_converter.cc:862][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [128] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.937.772 [graph_var_manager.cc:1424][EVENT]167129 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:35.937.877 [graph_manager.cc:1248][EVENT]167129 PreRun:PreRun start: graph node size 4, session id 18, graph id 17, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:35.938.148 [atrace_api.c:28](tid:167129) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:35.938.175 [trace_rb_log.c:84](tid:167129) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:35.938.188 [atrace_api.c:32](tid:167129) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:35.938.206 [client_manager.cpp:157][SetProfilingCallback][tid:167129] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:35.938.574 [parallel_partitioner.cc:165][EVENT]167129 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.938.644 [parallel_partitioner.cc:178][EVENT]167129 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.938.693 [graph_prepare.cc:1378][EVENT]167129 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.938.861 [graph_manager.cc:1050][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [185] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.938.887 [graph_manager.cc:1052][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.939.031 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.939.062 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.939.112 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.939.126 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.939.172 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.939.185 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.939.202 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.939.299 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.939.320 [graph_manager.cc:1054][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [421] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.939.550 [graph_manager.cc:1055][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [217] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.940.601 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:35.940.630 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.940.642 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.940.651 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [364] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.940.661 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.940.669 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [4] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:35.940.678 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.940.686 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.940.705 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [7] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.942.820 [graph_manager.cc:1056][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3249] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.942.890 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.942.909 [graph_prepare.cc:1982][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.943.354 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:35.943.380 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.391 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.400 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [255] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.409 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.418 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:35.943.426 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.434 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.443 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.468 [graph_prepare.cc:1983][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [546] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.943.492 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.943.503 [graph_prepare.cc:1984][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.943.517 [graph_prepare.cc:1985][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.943.531 [graph_prepare.cc:1986][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.943.542 [graph_prepare.cc:1987][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.943.557 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.943.569 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.943.582 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.943.685 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.702 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.710 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.719 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.728 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.739 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.750 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.759 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.767 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of StopGradientPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.776 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.784 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.796 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.804 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.812 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.820 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.829 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.943.850 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.943.863 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.943.897 [graph_prepare.cc:1988][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [345] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.943.910 [graph_manager.cc:1065][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1056] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.955.951 [graph_manager.cc:1077][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12021] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.956.024 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.956.076 [graph_manager.cc:1080][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [86] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.125 [graph_manager.cc:1081][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [5022] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.172 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.189 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.201 [graph_manager.cc:1082][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.237 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.251 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.266 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.295 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.309 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.323 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.335 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.375 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.394 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.413 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.440 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.455 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.467 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.477 [graph_manager.cc:2700][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [246] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.601 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.961.617 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.961.626 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.961.635 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.961.654 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.961.663 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CastRemovePass is [9] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.961.671 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.961.680 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.961.750 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.961.759 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.961.767 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.961.775 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.961.784 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.961.792 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.961.800 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.961.810 [graph_manager.cc:2741][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [310] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.819 [graph_manager.cc:2752][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.843 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.855 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.873 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.888 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.901 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.914 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.933 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.947 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.960 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.971 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.961.990 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.962.005 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.962.023 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.962.038 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.962.048 [graph_manager.cc:2810][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [211] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.962.078 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.962.092 [graph_manager.cc:2821][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.962.120 [graph_manager.cc:1087][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [897] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.962.256 [graph_manager.cc:1088][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [120] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.962.301 [graph_manager.cc:1089][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.962.325 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.962.340 [graph_manager.cc:1097][EVENT]167129 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:35.962.362 [graph_manager.cc:3325][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.962.896 [engine_place.cc:144][EVENT]167129 Run:The time cost of AIcoreEngine::CheckSupported is [431] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.962.928 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.962.940 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.963.018 [graph_manager.cc:3351][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [643] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.963.037 [graph_manager.cc:3364][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.963.108 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.963.127 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.963.286 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [149] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.963.346 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [33] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.963.394 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.963.427 [graph_manager.cc:3405][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [378] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.963.445 [graph_manager.cc:3412][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.016 [graph_manager.cc:3422][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [18556] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.061 [graph_manager.cc:3428][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.203 [graph_manager.cc:3467][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [120] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.223 [graph_manager.cc:3377][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [19174] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.239 [graph_manager.cc:1106][EVENT]167129 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [19883] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.251 [graph_manager.cc:1115][EVENT]167129 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:35.982.273 [graph_manager.cc:1130][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.305 [graph_manager.cc:1131][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.332 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.349 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.359 [graph_manager.cc:2837][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.441 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.982.454 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.982.464 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.982.473 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.982.481 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [7] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.982.490 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:35.982.511 [graph_manager.cc:2864][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [134] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.523 [graph_manager.cc:2872][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.544 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.558 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.574 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.588 [compile_nodes_pass.cc:88][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.597 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.607 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.690 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [74] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.719 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.733 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.746 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.759 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.769 [graph_manager.cc:2927][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [230] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.782 [graph_manager.cc:2937][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.796 [graph_manager.cc:2943][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.808 [graph_manager.cc:2950][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.982.982 [graph_manager.cc:2958][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.983.013 [graph_manager.cc:1132][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [694] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.983.084 [graph_manager.cc:1135][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [58] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.983.117 [graph_manager.cc:2975][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.983.158 [graph_manager.cc:2981][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.983.172 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.983.182 [graph_manager.cc:2986][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.983.192 [graph_manager.cc:1136][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [91] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.983.311 [graph_manager.cc:3555][EVENT]167129 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [87] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.983.401 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.983.418 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.983.537 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [109] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.983.570 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.983.608 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.983.631 [graph_builder.cc:865][EVENT]167129 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [265] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:35.983.922 [logger.cc:1071] 167129 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:37:35.983.952 [task_generator.cc:804][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [79] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.984.016 [task_generator.cc:805][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [50] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.984.701 [task_generator.cc:814][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [671] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.984.719 [task_generator.cc:954][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [848] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.984.782 [task_generator.cc:967][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [34] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:35.984.804 [logger.cc:1084] 167129 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:37:35.985.566 [graph_manager.cc:1152][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2347] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.985.603 [graph_manager.cc:1164][EVENT]167129 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:35.985.639 [graph_manager.cc:1271][EVENT]167129 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [47157] micro second. [INFO] GE(164040,python):2024-01-10-11:37:35.985.667 [graph_manager.cc:1272][EVENT]167129 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:37:35.986.041 [atrace_api.c:93](tid:167129) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:37:35.986.063 [atrace_api.c:95](tid:167129) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:37:35.999.827 [graph_converter.cc:838][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [4594] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.000.000 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [122] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.000.514 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [489] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.000.731 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [192] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.000.753 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [215] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.000.975 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [210] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.001.017 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.001.049 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.001.251 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [190] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.001.338 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [67] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.001.353 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [83] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.001.382 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.001.407 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.001.434 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.001.514 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [69] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.001.582 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [57] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.001.593 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [68] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.001.619 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.001.643 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.001.656 [graph_converter.cc:849][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1785] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.001.916 [graph_converter.cc:853][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [250] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.002.653 [graph_converter.cc:857][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [718] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.002.817 [graph_converter.cc:862][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [126] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.090.228 [graph_var_manager.cc:1424][EVENT]167132 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:36.090.330 [graph_manager.cc:1248][EVENT]167132 PreRun:PreRun start: graph node size 4, session id 19, graph id 18, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:36.091.252 [atrace_api.c:28](tid:167132) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:36.091.324 [trace_rb_log.c:84](tid:167132) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:36.091.339 [atrace_api.c:32](tid:167132) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:36.091.359 [client_manager.cpp:157][SetProfilingCallback][tid:167132] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:36.092.265 [parallel_partitioner.cc:165][EVENT]167132 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.092.306 [parallel_partitioner.cc:178][EVENT]167132 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.092.354 [graph_prepare.cc:1378][EVENT]167132 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.093.057 [graph_manager.cc:1050][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [719] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.093.086 [graph_manager.cc:1052][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.093.229 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.093.261 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.093.310 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.093.324 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.093.370 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.093.384 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.093.401 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.093.510 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.093.531 [graph_manager.cc:1054][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [432] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.093.780 [graph_manager.cc:1055][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [236] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.094.845 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:36.094.898 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.094.912 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.094.921 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of InferShapePass is [342] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.094.930 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [15] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.094.939 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:36.094.947 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [15] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.094.956 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.094.964 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of InferValuePass is [9] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.097.055 [graph_manager.cc:1056][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3251] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.097.127 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.097.147 [graph_prepare.cc:1982][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.097.597 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:36.097.625 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.097.637 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.097.646 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of InferShapePass is [249] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.097.655 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.097.664 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:36.097.676 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.097.684 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.097.727 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.097.754 [graph_prepare.cc:1983][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [593] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.097.780 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.097.802 [graph_prepare.cc:1984][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [30] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.097.820 [graph_prepare.cc:1985][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.097.842 [graph_prepare.cc:1986][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.097.854 [graph_prepare.cc:1987][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.097.869 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.097.883 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.097.902 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.098.000 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.098.018 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.098.029 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.098.042 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.098.051 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.098.060 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.098.069 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.098.077 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.098.085 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.098.093 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.098.102 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.098.110 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.098.118 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.098.126 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.098.135 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.098.143 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.098.172 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.098.184 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.098.217 [graph_prepare.cc:1988][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [354] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.098.230 [graph_manager.cc:1065][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1138] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.110.521 [graph_manager.cc:1077][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12270] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.110.592 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.110.642 [graph_manager.cc:1080][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [82] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.106 [graph_manager.cc:1081][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [8446] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.153 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.170 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.181 [graph_manager.cc:1082][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.214 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.230 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.245 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.341 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [86] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.359 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.403 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [34] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.418 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.457 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [29] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.477 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.504 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.543 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.559 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.572 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.581 [graph_manager.cc:2700][EVENT]167132 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [372] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.706 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.119.720 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.119.730 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.119.739 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.119.747 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.119.756 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of CastRemovePass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.119.764 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.119.773 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.119.781 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.119.789 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.119.798 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.119.806 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.119.814 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.119.822 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.119.830 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.119.840 [graph_manager.cc:2741][EVENT]167132 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [241] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.849 [graph_manager.cc:2752][EVENT]167132 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.872 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.883 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.909 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.923 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.934 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.947 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.965 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.979 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.119.992 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.120.002 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.120.014 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.120.025 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.120.043 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.120.055 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.120.064 [graph_manager.cc:2810][EVENT]167132 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [197] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.120.093 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.120.104 [graph_manager.cc:2821][EVENT]167132 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.120.133 [graph_manager.cc:1087][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [932] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.120.266 [graph_manager.cc:1088][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [120] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.120.306 [graph_manager.cc:1089][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.120.325 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.120.340 [graph_manager.cc:1097][EVENT]167132 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:36.120.362 [graph_manager.cc:3325][EVENT]167132 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.120.750 [engine_place.cc:144][EVENT]167132 Run:The time cost of AIcoreEngine::CheckSupported is [292] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.120.793 [engine_place.cc:144][EVENT]167132 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.120.803 [engine_place.cc:144][EVENT]167132 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.120.885 [graph_manager.cc:3351][EVENT]167132 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [510] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.120.904 [graph_manager.cc:3364][EVENT]167132 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.120.978 [engine_partitioner.cc:1139][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.120.996 [engine_partitioner.cc:1142][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.121.162 [engine_partitioner.cc:1148][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [156] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.121.209 [engine_partitioner.cc:1155][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.121.260 [engine_partitioner.cc:1164][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.121.294 [graph_manager.cc:3405][EVENT]167132 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [379] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.121.313 [graph_manager.cc:3412][EVENT]167132 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.176 [graph_manager.cc:3422][EVENT]167132 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [16850] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.217 [graph_manager.cc:3428][EVENT]167132 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.351 [graph_manager.cc:3467][EVENT]167132 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [115] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.371 [graph_manager.cc:3377][EVENT]167132 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [17455] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.387 [graph_manager.cc:1106][EVENT]167132 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [18032] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.399 [graph_manager.cc:1115][EVENT]167132 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:36.138.422 [graph_manager.cc:1130][EVENT]167132 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.454 [graph_manager.cc:1131][EVENT]167132 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.478 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.506 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.516 [graph_manager.cc:2837][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [47] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.599 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.138.613 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.138.622 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.138.631 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.138.640 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.138.648 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [7] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.138.658 [graph_manager.cc:2864][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [125] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.669 [graph_manager.cc:2872][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.689 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.702 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.718 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.731 [compile_nodes_pass.cc:88][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.741 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.751 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.832 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [72] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.860 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.873 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.886 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.899 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.909 [graph_manager.cc:2927][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [223] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.929 [graph_manager.cc:2937][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.943 [graph_manager.cc:2943][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.138.954 [graph_manager.cc:2950][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.139.126 [graph_manager.cc:2958][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.139.158 [graph_manager.cc:1132][EVENT]167132 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [690] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.139.224 [graph_manager.cc:1135][EVENT]167132 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [53] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.139.262 [graph_manager.cc:2975][EVENT]167132 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.139.293 [graph_manager.cc:2981][EVENT]167132 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.139.307 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.139.317 [graph_manager.cc:2986][EVENT]167132 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.139.326 [graph_manager.cc:1136][EVENT]167132 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [87] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.139.456 [graph_manager.cc:3555][EVENT]167132 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [98] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.139.550 [engine_partitioner.cc:1139][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.139.567 [engine_partitioner.cc:1142][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.139.704 [engine_partitioner.cc:1148][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [126] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.139.741 [engine_partitioner.cc:1155][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.139.782 [engine_partitioner.cc:1164][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [29] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.139.805 [graph_builder.cc:865][EVENT]167132 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [292] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:36.140.254 [logger.cc:1071] 167132 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:37:36.140.291 [task_generator.cc:804][EVENT]167132 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [177] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.140.356 [task_generator.cc:805][EVENT]167132 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [52] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.141.087 [task_generator.cc:814][EVENT]167132 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [706] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.141.103 [task_generator.cc:954][EVENT]167132 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [990] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.141.164 [task_generator.cc:967][EVENT]167132 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [35] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:36.141.182 [logger.cc:1084] 167132 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:37:36.142.497 [graph_manager.cc:1152][EVENT]167132 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3145] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.142.536 [graph_manager.cc:1164][EVENT]167132 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:36.142.572 [graph_manager.cc:1271][EVENT]167132 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [50398] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.142.588 [graph_manager.cc:1272][EVENT]167132 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:37:36.142.903 [atrace_api.c:93](tid:167132) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:37:36.142.923 [atrace_api.c:95](tid:167132) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:37:36.154.278 [graph_converter.cc:838][EVENT]167132 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [3542] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.154.453 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of ZeroCopy is [125] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.154.959 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of CEM is [481] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.155.176 [copy_flow_launch_fuse.cc:395][EVENT]167132 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [191] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.155.201 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [217] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.155.423 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [206] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.155.467 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.155.501 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.155.705 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of CEM is [188] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.155.793 [copy_flow_launch_fuse.cc:395][EVENT]167132 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [66] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.155.812 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [85] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.155.844 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.155.873 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.155.903 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.155.985 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of CEM is [70] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.156.073 [copy_flow_launch_fuse.cc:395][EVENT]167132 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [59] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.156.087 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [73] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.156.117 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.156.144 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.156.158 [graph_converter.cc:849][EVENT]167132 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1836] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.156.385 [graph_converter.cc:853][EVENT]167132 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [218] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.157.137 [graph_converter.cc:857][EVENT]167132 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [735] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.157.292 [graph_converter.cc:862][EVENT]167132 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [126] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.245.811 [graph_var_manager.cc:1424][EVENT]167130 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:36.245.914 [graph_manager.cc:1248][EVENT]167130 PreRun:PreRun start: graph node size 4, session id 20, graph id 19, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:36.246.206 [atrace_api.c:28](tid:167130) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:36.246.237 [trace_rb_log.c:84](tid:167130) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:36.246.251 [atrace_api.c:32](tid:167130) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:36.246.267 [client_manager.cpp:157][SetProfilingCallback][tid:167130] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:36.246.683 [parallel_partitioner.cc:165][EVENT]167130 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.246.720 [parallel_partitioner.cc:178][EVENT]167130 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.246.766 [graph_prepare.cc:1378][EVENT]167130 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.247.375 [graph_manager.cc:1050][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [625] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.247.406 [graph_manager.cc:1052][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.247.553 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.247.588 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.247.637 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.247.679 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.247.725 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.247.739 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.247.758 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.247.861 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.247.882 [graph_manager.cc:1054][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [461] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.248.108 [graph_manager.cc:1055][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [212] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.249.286 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:36.249.318 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.249.329 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.249.339 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [460] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.249.347 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [15] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.249.356 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:36.249.365 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.249.373 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.249.381 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.252.346 [graph_manager.cc:1056][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [4216] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.252.416 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.252.435 [graph_prepare.cc:1982][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [52] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.252.977 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:36.253.006 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.017 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.027 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [340] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.047 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.057 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:37:36.253.066 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.075 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.083 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.109 [graph_prepare.cc:1983][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [661] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.253.133 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.253.145 [graph_prepare.cc:1984][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.253.159 [graph_prepare.cc:1985][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.253.173 [graph_prepare.cc:1986][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.253.184 [graph_prepare.cc:1987][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.253.199 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.253.211 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.253.225 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.253.320 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.332 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.341 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.350 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.359 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DropOutPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.367 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.375 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.384 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.392 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.407 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.416 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.425 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.433 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.441 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.450 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.458 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:37:36.253.481 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.253.494 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.253.527 [graph_prepare.cc:1988][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [333] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.253.540 [graph_manager.cc:1065][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1158] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.266.536 [graph_manager.cc:1077][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12976] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.266.637 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.266.688 [graph_manager.cc:1080][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [112] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.278.012 [graph_manager.cc:1081][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [11305] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.278.060 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.278.077 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.278.090 [graph_manager.cc:1082][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.278.122 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.278.137 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.278.152 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.278.317 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [142] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.278.336 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.278.426 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [79] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.278.441 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.278.491 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [40] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.278.513 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.278.537 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.278.627 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [79] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.278.648 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.278.661 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.278.671 [graph_manager.cc:2700][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [555] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.278.911 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.278.926 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.278.936 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.278.945 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [4] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.278.954 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.278.963 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CastRemovePass is [41] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.278.971 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [6] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.278.980 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.278.988 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [12] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.278.996 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [4] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.279.005 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [17] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.279.013 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.279.037 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.279.046 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [11] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.279.054 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.279.064 [graph_manager.cc:2741][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [373] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.279.073 [graph_manager.cc:2752][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.279.098 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.279.110 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.279.132 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.279.149 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.279.161 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.279.174 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.279.195 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.279.210 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.279.223 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.279.233 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.279.245 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.279.256 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.279.279 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.279.293 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.279.302 [graph_manager.cc:2810][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [209] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.279.346 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.279.359 [graph_manager.cc:2821][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [49] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.279.396 [graph_manager.cc:1087][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1288] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.279.957 [graph_manager.cc:1088][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [546] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.280.022 [graph_manager.cc:1089][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.280.045 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.280.062 [graph_manager.cc:1097][EVENT]167130 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:36.280.085 [graph_manager.cc:3325][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.289.756 [engine_place.cc:144][EVENT]167130 Run:The time cost of AIcoreEngine::CheckSupported is [9454] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.289.791 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.289.802 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.289.896 [graph_manager.cc:3351][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [9796] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.289.916 [graph_manager.cc:3364][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.289.994 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [29] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.290.027 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.290.207 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [169] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.290.250 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.290.302 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [40] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.290.339 [graph_manager.cc:3405][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [411] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.290.357 [graph_manager.cc:3412][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.405.534 [graph_manager.cc:3422][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [115161] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.405.586 [graph_manager.cc:3428][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.405.817 [graph_manager.cc:3467][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [208] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.405.853 [graph_manager.cc:3377][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [115924] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.405.871 [graph_manager.cc:1106][EVENT]167130 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [125793] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.405.884 [graph_manager.cc:1115][EVENT]167130 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:36.405.910 [graph_manager.cc:1130][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.405.944 [graph_manager.cc:1131][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.405.972 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.405.992 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.002 [graph_manager.cc:2837][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [43] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.142 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [28] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.406.157 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.406.166 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.406.175 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.406.184 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [10] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.406.193 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [15] micro second, call num is [9] [INFO] GE(164040,python):2024-01-10-11:37:36.406.203 [graph_manager.cc:2864][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [182] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.215 [graph_manager.cc:2872][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.235 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.250 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.268 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.282 [compile_nodes_pass.cc:88][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.292 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.311 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.422 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [101] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.471 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.485 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.499 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.513 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.523 [graph_manager.cc:2927][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [293] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.535 [graph_manager.cc:2937][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.551 [graph_manager.cc:2943][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.561 [graph_manager.cc:2950][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.761 [graph_manager.cc:2958][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [56] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.796 [graph_manager.cc:1132][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [838] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.874 [graph_manager.cc:1135][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [65] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.912 [graph_manager.cc:2975][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.945 [graph_manager.cc:2981][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.959 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.970 [graph_manager.cc:2986][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.406.979 [graph_manager.cc:1136][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [87] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.407.309 [graph_manager.cc:3555][EVENT]167130 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [291] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.407.439 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.407.470 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.407.641 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [144] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.407.680 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [25] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.407.723 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.407.751 [graph_builder.cc:865][EVENT]167130 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [374] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:36.408.292 [logger.cc:1071] 167130 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:37:36.408.329 [task_generator.cc:804][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [184] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.408.419 [task_generator.cc:805][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [77] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.410.210 [task_generator.cc:814][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1773] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.410.228 [task_generator.cc:954][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2083] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.410.301 [task_generator.cc:967][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [39] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:37:36.410.326 [logger.cc:1084] 167130 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:37:36.413.314 [graph_manager.cc:1152][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [6301] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.413.355 [graph_manager.cc:1164][EVENT]167130 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:37:36.413.396 [graph_manager.cc:1271][EVENT]167130 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [166803] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.413.408 [graph_manager.cc:1272][EVENT]167130 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:37:36.413.763 [atrace_api.c:93](tid:167130) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:37:36.413.782 [atrace_api.c:95](tid:167130) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:37:36.441.569 [graph_converter.cc:838][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [10957] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.441.839 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [211] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.443.409 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [1537] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.443.815 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [371] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.443.847 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [406] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.444.117 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [254] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.444.207 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [61] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.444.295 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [53] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.444.767 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [453] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.444.988 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [191] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.445.013 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [217] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.445.079 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [54] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.445.139 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [45] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.445.200 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [47] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.445.428 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [215] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.445.622 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [176] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.445.642 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [197] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.445.730 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [77] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.445.793 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [46] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.445.813 [graph_converter.cc:849][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4193] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.446.527 [graph_converter.cc:853][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [701] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.448.497 [graph_converter.cc:857][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1941] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.448.900 [graph_converter.cc:862][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [369] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.534.087 [graph_var_manager.cc:1424][EVENT]167132 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:37:36.534.202 [graph_manager.cc:1248][EVENT]167132 PreRun:PreRun start: graph node size 5, session id 21, graph id 20, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:37:36.534.447 [atrace_api.c:28](tid:167132) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:37:36.534.478 [trace_rb_log.c:84](tid:167132) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:37:36.534.492 [atrace_api.c:32](tid:167132) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:37:36.534.510 [client_manager.cpp:157][SetProfilingCallback][tid:167132] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:37:36.534.926 [parallel_partitioner.cc:165][EVENT]167132 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.534.988 [parallel_partitioner.cc:178][EVENT]167132 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.535.038 [graph_prepare.cc:1378][EVENT]167132 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.535.175 [graph_manager.cc:1050][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [155] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.535.200 [graph_manager.cc:1052][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.535.354 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.535.387 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.535.435 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.535.448 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.535.493 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.535.507 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.535.524 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.535.628 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.535.650 [graph_manager.cc:1054][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [435] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.535.873 [graph_manager.cc:1055][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [211] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.537.094 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of AssertPass is [5] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:36.537.126 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.537.139 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.537.148 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of InferShapePass is [423] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.537.157 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.537.166 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [5] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:36.537.175 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [10] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.537.183 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [22] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.537.202 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.539.614 [graph_manager.cc:1056][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3719] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.539.687 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.539.707 [graph_prepare.cc:1982][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [56] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.540.296 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:36.540.324 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.335 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.345 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of InferShapePass is [331] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.354 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.362 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [4] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:37:36.540.371 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [9] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.380 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.389 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.442 [graph_prepare.cc:1983][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [721] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.540.468 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.540.479 [graph_prepare.cc:1984][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.540.493 [graph_prepare.cc:1985][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.540.507 [graph_prepare.cc:1986][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.540.518 [graph_prepare.cc:1987][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.540.533 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.540.547 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.540.560 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.540.664 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.688 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.697 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.706 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.715 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of DropOutPass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.724 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.732 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.740 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.749 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of StopGradientPass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.757 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.765 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.774 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.782 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.790 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [7] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.799 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.807 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:37:36.540.832 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.540.846 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.540.881 [graph_prepare.cc:1988][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [354] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.540.895 [graph_manager.cc:1065][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1245] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.552.939 [graph_manager.cc:1077][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12023] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.553.047 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.553.101 [graph_manager.cc:1080][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [122] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.562.076 [graph_manager.cc:1081][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [8946] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.562.124 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.562.141 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.562.154 [graph_manager.cc:1082][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.562.188 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.562.204 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.562.219 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.562.403 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [174] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.562.422 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.562.541 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [110] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.562.560 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.562.616 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [45] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.562.638 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.562.660 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.562.768 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [98] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.562.786 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.562.800 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.562.810 [graph_manager.cc:2700][EVENT]167132 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [628] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.563.096 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:37:36.563.114 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of AddNPass is [4] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:37:36.563.124 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:37:36.563.133 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [4] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:37:36.563.152 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:37:36.563.162 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of CastRemovePass is [50] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:37:36.563.170 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [6] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:37:36.563.179 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:37:36.563.187 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [14] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:37:36.563.195 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:37:36.563.204 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [22] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:37:36.563.212 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [15] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:37:36.563.220 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [23] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:37:36.563.228 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [10] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:37:36.563.237 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:37:36.563.247 [graph_manager.cc:2741][EVENT]167132 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [418] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.563.256 [graph_manager.cc:2752][EVENT]167132 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.563.281 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.563.294 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.563.318 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.563.335 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.563.347 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.563.360 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.563.382 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.563.397 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.563.411 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.563.421 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.563.439 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.563.452 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.563.478 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.563.492 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.563.501 [graph_manager.cc:2810][EVENT]167132 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [226] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.563.554 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of IdentityPass is [6] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:37:36.563.567 [graph_manager.cc:2821][EVENT]167132 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [56] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.563.596 [graph_manager.cc:1087][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1422] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.564.284 [graph_manager.cc:1088][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [674] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.564.353 [graph_manager.cc:1089][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.564.378 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.564.397 [graph_manager.cc:1097][EVENT]167132 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:37:36.564.420 [graph_manager.cc:3325][EVENT]167132 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.575.167 [engine_place.cc:144][EVENT]167132 Run:The time cost of AIcoreEngine::CheckSupported is [10476] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.575.201 [engine_place.cc:144][EVENT]167132 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.575.212 [engine_place.cc:144][EVENT]167132 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.575.312 [graph_manager.cc:3351][EVENT]167132 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10878] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.575.332 [graph_manager.cc:3364][EVENT]167132 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.575.420 [engine_partitioner.cc:1139][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [33] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.575.455 [engine_partitioner.cc:1142][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.575.663 [engine_partitioner.cc:1148][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [198] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.575.727 [engine_partitioner.cc:1155][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.575.778 [engine_partitioner.cc:1164][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.575.816 [graph_manager.cc:3405][EVENT]167132 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [471] micro second. [INFO] GE(164040,python):2024-01-10-11:37:36.575.836 [graph_manager.cc:3412][EVENT]167132 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. - \ | / - \ | / - \ | [INFO] GE(164040,python):2024-01-10-11:39:28.564.420 [graph_manager.cc:3422][EVENT]167132 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [111988569] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.564.524 [graph_manager.cc:3428][EVENT]167132 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.564.798 [graph_manager.cc:3467][EVENT]167132 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [244] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.564.821 [graph_manager.cc:3377][EVENT]167132 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [111989477] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.564.842 [graph_manager.cc:1106][EVENT]167132 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [112000430] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.564.855 [graph_manager.cc:1115][EVENT]167132 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:39:28.564.889 [graph_manager.cc:1130][EVENT]167132 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.564.930 [graph_manager.cc:1131][EVENT]167132 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.564.965 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.564.990 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.565.001 [graph_manager.cc:2837][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [53] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.565.201 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [44] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:28.565.217 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [6] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:28.565.227 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:28.565.236 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of BitcastPass is [4] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:28.565.245 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [10] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:28.565.254 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [19] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:28.565.281 [graph_manager.cc:2864][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [259] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.565.295 [graph_manager.cc:2872][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.565.317 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.565.333 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.565.352 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.565.369 [compile_nodes_pass.cc:88][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.565.379 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.565.389 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.565.526 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [127] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.565.631 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [90] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.565.647 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.565.663 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.565.681 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.565.734 [graph_manager.cc:2927][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [423] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.565.749 [graph_manager.cc:2937][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.565.766 [graph_manager.cc:2943][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.565.778 [graph_manager.cc:2950][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.566.131 [graph_manager.cc:2958][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [69] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.566.168 [graph_manager.cc:1132][EVENT]167132 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [1224] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.566.287 [graph_manager.cc:1135][EVENT]167132 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [105] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.566.346 [graph_manager.cc:2975][EVENT]167132 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [30] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.566.378 [graph_manager.cc:2981][EVENT]167132 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.566.392 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.566.403 [graph_manager.cc:2986][EVENT]167132 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.566.412 [graph_manager.cc:1136][EVENT]167132 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [98] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.566.816 [graph_manager.cc:3555][EVENT]167132 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [353] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.566.988 [engine_partitioner.cc:1139][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [43] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.567.029 [engine_partitioner.cc:1142][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.567.237 [engine_partitioner.cc:1148][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [197] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.567.284 [engine_partitioner.cc:1155][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.567.337 [engine_partitioner.cc:1164][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [40] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.567.366 [graph_builder.cc:865][EVENT]167132 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [458] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:39:28.567.950 [logger.cc:1071] 167132 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:39:28.567.990 [task_generator.cc:804][EVENT]167132 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [162] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.568.099 [task_generator.cc:805][EVENT]167132 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [96] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.570.361 [task_generator.cc:814][EVENT]167132 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [2246] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.570.380 [task_generator.cc:954][EVENT]167132 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2552] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.570.459 [task_generator.cc:967][EVENT]167132 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [43] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:39:28.570.485 [logger.cc:1084] 167132 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:39:28.572.536 [graph_manager.cc:1152][EVENT]167132 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [6085] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.572.575 [graph_manager.cc:1164][EVENT]167132 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:39:28.572.621 [graph_manager.cc:1271][EVENT]167132 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [112037787] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.572.644 [graph_manager.cc:1272][EVENT]167132 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:39:28.573.060 [atrace_api.c:93](tid:167132) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:39:28.573.081 [atrace_api.c:95](tid:167132) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:39:28.603.871 [graph_converter.cc:838][EVENT]167132 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [12216] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.604.124 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of ZeroCopy is [197] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.606.205 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of CEM is [2052] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.606.738 [copy_flow_launch_fuse.cc:395][EVENT]167132 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [500] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.606.766 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [531] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.607.070 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [290] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.607.175 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [79] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.607.259 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of ZeroCopy is [63] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.607.862 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of CEM is [586] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.608.130 [copy_flow_launch_fuse.cc:395][EVENT]167132 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [238] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.608.155 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [265] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.608.234 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [68] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.608.307 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [57] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.608.379 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of ZeroCopy is [59] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.608.664 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of CEM is [271] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.608.906 [copy_flow_launch_fuse.cc:395][EVENT]167132 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [222] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.608.927 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [243] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.609.002 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [65] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.609.072 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [56] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.609.093 [graph_converter.cc:849][EVENT]167132 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [5172] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.610.086 [graph_converter.cc:853][EVENT]167132 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [982] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.612.543 [graph_converter.cc:857][EVENT]167132 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2428] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.613.069 [graph_converter.cc:862][EVENT]167132 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [480] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.700.025 [graph_var_manager.cc:1424][EVENT]167132 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:39:28.700.135 [graph_manager.cc:1248][EVENT]167132 PreRun:PreRun start: graph node size 4, session id 22, graph id 21, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:39:28.700.417 [atrace_api.c:28](tid:167132) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:39:28.700.446 [trace_rb_log.c:84](tid:167132) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:39:28.700.459 [atrace_api.c:32](tid:167132) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:39:28.700.483 [client_manager.cpp:157][SetProfilingCallback][tid:167132] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:39:28.700.865 [parallel_partitioner.cc:165][EVENT]167132 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.700.904 [parallel_partitioner.cc:178][EVENT]167132 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.700.951 [graph_prepare.cc:1378][EVENT]167132 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.701.099 [graph_manager.cc:1050][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [166] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.701.124 [graph_manager.cc:1052][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.701.268 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.701.299 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.701.349 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.701.362 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.701.406 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.701.419 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.701.436 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.701.552 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.701.574 [graph_manager.cc:1054][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [436] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.701.848 [graph_manager.cc:1055][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [261] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.702.926 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:39:28.702.955 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.702.967 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.702.977 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of InferShapePass is [346] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.702.986 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.702.995 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:39:28.703.004 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.703.012 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.703.021 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.705.243 [graph_manager.cc:1056][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3343] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.705.311 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.705.331 [graph_prepare.cc:1982][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.705.846 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:39:28.705.876 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.705.887 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.705.897 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of InferShapePass is [288] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.705.906 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.705.915 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:39:28.705.923 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.705.932 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.705.940 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.705.988 [graph_prepare.cc:1983][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [644] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.706.013 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.706.035 [graph_prepare.cc:1984][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.706.050 [graph_prepare.cc:1985][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.706.064 [graph_prepare.cc:1986][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.706.076 [graph_prepare.cc:1987][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.706.091 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.706.103 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.706.118 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.706.211 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.706.223 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.706.233 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.706.242 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.706.250 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of DropOutPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.706.259 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.706.267 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.706.276 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.706.284 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.706.293 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.706.301 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.706.310 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.706.318 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.706.326 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.706.334 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.706.343 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.706.372 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.706.386 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.706.418 [graph_prepare.cc:1988][EVENT]167132 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [332] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.706.430 [graph_manager.cc:1065][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1154] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.719.817 [graph_manager.cc:1077][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13366] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.719.892 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.719.943 [graph_manager.cc:1080][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [88] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.172 [graph_manager.cc:1081][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [5213] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.214 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.228 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.240 [graph_manager.cc:1082][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [34] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.272 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.286 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.302 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.338 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.351 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.365 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.378 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.419 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.437 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.456 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.498 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.513 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.525 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.534 [graph_manager.cc:2700][EVENT]167132 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [268] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.656 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.725.670 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.725.680 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.725.718 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.725.727 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.725.736 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of CastRemovePass is [9] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.725.745 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.725.753 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.725.762 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.725.770 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.725.778 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.725.786 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.725.795 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.725.803 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.725.811 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.725.821 [graph_manager.cc:2741][EVENT]167132 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [267] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.830 [graph_manager.cc:2752][EVENT]167132 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.854 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.866 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.892 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.908 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.920 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.933 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.953 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.968 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.981 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.725.992 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.726.005 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.726.016 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.726.035 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.726.049 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.726.058 [graph_manager.cc:2810][EVENT]167132 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [209] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.726.089 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.726.100 [graph_manager.cc:2821][EVENT]167132 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [33] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.726.128 [graph_manager.cc:1087][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [869] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.726.261 [graph_manager.cc:1088][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [121] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.726.302 [graph_manager.cc:1089][EVENT]167132 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.726.321 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.726.335 [graph_manager.cc:1097][EVENT]167132 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:39:28.726.357 [graph_manager.cc:3325][EVENT]167132 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.726.878 [engine_place.cc:144][EVENT]167132 Run:The time cost of AIcoreEngine::CheckSupported is [400] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.726.918 [engine_place.cc:144][EVENT]167132 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.726.928 [engine_place.cc:144][EVENT]167132 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.727.009 [graph_manager.cc:3351][EVENT]167132 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [639] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.727.027 [graph_manager.cc:3364][EVENT]167132 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.727.102 [engine_partitioner.cc:1139][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.727.119 [engine_partitioner.cc:1142][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.727.282 [engine_partitioner.cc:1148][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [153] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.727.328 [engine_partitioner.cc:1155][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.727.378 [engine_partitioner.cc:1164][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.727.413 [graph_manager.cc:3405][EVENT]167132 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [373] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.727.431 [graph_manager.cc:3412][EVENT]167132 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.744.471 [graph_manager.cc:3422][EVENT]167132 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [17025] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.744.515 [graph_manager.cc:3428][EVENT]167132 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.744.657 [graph_manager.cc:3467][EVENT]167132 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [120] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.744.677 [graph_manager.cc:3377][EVENT]167132 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [17638] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.744.694 [graph_manager.cc:1106][EVENT]167132 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [18345] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.744.707 [graph_manager.cc:1115][EVENT]167132 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:39:28.744.730 [graph_manager.cc:1130][EVENT]167132 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.744.761 [graph_manager.cc:1131][EVENT]167132 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.744.785 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.744.814 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.744.824 [graph_manager.cc:2837][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [47] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.744.904 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.744.916 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.744.925 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of CondRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.744.934 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.744.943 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [7] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.744.951 [base_pass.cc:339][EVENT]167132 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.744.961 [graph_manager.cc:2864][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [120] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.744.972 [graph_manager.cc:2872][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.744.992 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.006 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.022 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.035 [compile_nodes_pass.cc:88][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.044 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.054 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.137 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [73] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.167 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.180 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.193 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.206 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.215 [graph_manager.cc:2927][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [226] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.235 [graph_manager.cc:2937][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.249 [graph_manager.cc:2943][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.260 [graph_manager.cc:2950][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.434 [graph_manager.cc:2958][EVENT]167132 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.467 [graph_manager.cc:1132][EVENT]167132 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [692] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.539 [graph_manager.cc:1135][EVENT]167132 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [58] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.570 [graph_manager.cc:2975][EVENT]167132 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.603 [graph_manager.cc:2981][EVENT]167132 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.618 [pass_manager.cc:82][EVENT]167132 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.629 [graph_manager.cc:2986][EVENT]167132 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.639 [graph_manager.cc:1136][EVENT]167132 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [84] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.828 [graph_manager.cc:3555][EVENT]167132 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [156] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.923 [engine_partitioner.cc:1139][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.745.940 [engine_partitioner.cc:1142][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.746.061 [engine_partitioner.cc:1148][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [111] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.746.093 [engine_partitioner.cc:1155][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.746.132 [engine_partitioner.cc:1164][EVENT]167132 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.746.154 [graph_builder.cc:865][EVENT]167132 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [266] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:39:28.746.451 [logger.cc:1071] 167132 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:39:28.746.482 [task_generator.cc:804][EVENT]167132 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [83] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.746.543 [task_generator.cc:805][EVENT]167132 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [49] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.747.281 [task_generator.cc:814][EVENT]167132 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [714] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.747.296 [task_generator.cc:954][EVENT]167132 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [898] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.747.357 [task_generator.cc:967][EVENT]167132 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [34] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:39:28.747.375 [logger.cc:1084] 167132 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:39:28.748.166 [graph_manager.cc:1152][EVENT]167132 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2500] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.748.200 [graph_manager.cc:1164][EVENT]167132 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:39:28.748.236 [graph_manager.cc:1271][EVENT]167132 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [47465] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.748.247 [graph_manager.cc:1272][EVENT]167132 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:39:28.748.566 [atrace_api.c:93](tid:167132) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:39:28.748.581 [atrace_api.c:95](tid:167132) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:39:28.762.803 [graph_converter.cc:838][EVENT]167132 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [4697] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.762.978 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of ZeroCopy is [124] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.763.490 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of CEM is [487] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.763.707 [copy_flow_launch_fuse.cc:395][EVENT]167132 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [191] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.763.729 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [214] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.763.954 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [213] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.763.996 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.764.028 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.764.229 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of CEM is [188] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.764.315 [copy_flow_launch_fuse.cc:395][EVENT]167132 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [67] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.764.329 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [82] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.764.359 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.764.385 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.764.411 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.764.492 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of CEM is [70] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.764.579 [copy_flow_launch_fuse.cc:395][EVENT]167132 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [60] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.764.592 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [73] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.764.618 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.764.643 [base_optimizer.cc:70][EVENT]167132 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.764.657 [graph_converter.cc:849][EVENT]167132 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1809] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.764.887 [graph_converter.cc:853][EVENT]167132 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [221] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.765.631 [graph_converter.cc:857][EVENT]167132 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [728] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.765.819 [graph_converter.cc:862][EVENT]167132 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [162] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.841.046 [graph_var_manager.cc:1424][EVENT]167130 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:39:28.841.147 [graph_manager.cc:1248][EVENT]167130 PreRun:PreRun start: graph node size 4, session id 23, graph id 22, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:39:28.841.413 [atrace_api.c:28](tid:167130) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:39:28.841.444 [trace_rb_log.c:84](tid:167130) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:39:28.841.457 [atrace_api.c:32](tid:167130) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:39:28.841.474 [client_manager.cpp:157][SetProfilingCallback][tid:167130] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:39:28.841.897 [parallel_partitioner.cc:165][EVENT]167130 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.841.938 [parallel_partitioner.cc:178][EVENT]167130 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.841.987 [graph_prepare.cc:1378][EVENT]167130 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.842.164 [graph_manager.cc:1050][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [196] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.842.189 [graph_manager.cc:1052][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.842.331 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.842.362 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.842.408 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.842.444 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.842.493 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.842.505 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.842.522 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.842.629 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.842.650 [graph_manager.cc:1054][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [447] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.842.867 [graph_manager.cc:1055][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [203] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.843.899 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:39:28.843.929 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.843.941 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.843.951 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [325] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.843.960 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.843.969 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:39:28.843.977 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.843.986 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.843.994 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.846.090 [graph_manager.cc:1056][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3204] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.846.160 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.846.179 [graph_prepare.cc:1982][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.846.597 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:39:28.846.624 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.846.635 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.846.644 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [225] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.846.663 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.846.673 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:39:28.846.681 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.846.690 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.846.698 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.846.724 [graph_prepare.cc:1983][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [532] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.846.747 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.846.758 [graph_prepare.cc:1984][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.846.772 [graph_prepare.cc:1985][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.846.787 [graph_prepare.cc:1986][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.846.797 [graph_prepare.cc:1987][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.846.811 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.846.823 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.846.837 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.846.931 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.846.943 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.846.952 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.846.961 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.846.969 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.846.977 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.846.986 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.846.994 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.847.011 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.847.019 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.847.028 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.847.036 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.847.044 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.847.052 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.847.060 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.847.069 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.847.092 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.847.106 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.847.140 [graph_prepare.cc:1988][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [334] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.847.153 [graph_manager.cc:1065][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1027] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.859.844 [graph_manager.cc:1077][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12671] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.859.907 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.859.957 [graph_manager.cc:1080][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [76] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.863.431 [graph_manager.cc:1081][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3458] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.863.474 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.863.489 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.863.500 [graph_manager.cc:1082][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [34] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.863.531 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.863.545 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.863.559 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.863.603 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.863.617 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.863.632 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.863.646 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.863.687 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.863.706 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.863.736 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.863.765 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.863.780 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.863.792 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.863.801 [graph_manager.cc:2700][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [275] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.863.921 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.863.934 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.863.943 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.863.952 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.863.961 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.863.969 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CastRemovePass is [9] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.863.977 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.863.986 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.863.994 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.864.002 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.864.011 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.864.019 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.864.035 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.864.044 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.864.052 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.864.061 [graph_manager.cc:2741][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [242] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.070 [graph_manager.cc:2752][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.093 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.105 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.122 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.137 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.148 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.160 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.179 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.192 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.205 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.215 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.234 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.245 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.263 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.274 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.283 [graph_manager.cc:2810][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [194] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.313 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:28.864.324 [graph_manager.cc:2821][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.358 [graph_manager.cc:1087][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [839] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.490 [graph_manager.cc:1088][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [120] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.531 [graph_manager.cc:1089][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.548 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.576 [graph_manager.cc:1097][EVENT]167130 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:39:28.864.598 [graph_manager.cc:3325][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.864.979 [engine_place.cc:144][EVENT]167130 Run:The time cost of AIcoreEngine::CheckSupported is [276] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.865.007 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.865.017 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.865.103 [graph_manager.cc:3351][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [493] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.865.123 [graph_manager.cc:3364][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.865.199 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.865.217 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.865.374 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [147] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.865.421 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.865.470 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.865.507 [graph_manager.cc:3405][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [372] micro second. [INFO] GE(164040,python):2024-01-10-11:39:28.865.525 [graph_manager.cc:3412][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.035.809 [graph_manager.cc:3422][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [170271] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.035.860 [graph_manager.cc:3428][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.013 [graph_manager.cc:3467][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [132] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.046 [graph_manager.cc:3377][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [170911] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.063 [graph_manager.cc:1106][EVENT]167130 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [171472] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.076 [graph_manager.cc:1115][EVENT]167130 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:39:29.036.099 [graph_manager.cc:1130][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.134 [graph_manager.cc:1131][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.158 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.175 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.185 [graph_manager.cc:2837][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.274 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:29.036.287 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:29.036.296 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:29.036.305 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of BitcastPass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:29.036.313 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:29.036.322 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:39:29.036.332 [graph_manager.cc:2864][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [129] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.344 [graph_manager.cc:2872][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.365 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.379 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.395 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.408 [compile_nodes_pass.cc:88][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.417 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.436 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.522 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [76] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.551 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.564 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.577 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.591 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.600 [graph_manager.cc:2927][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [239] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.612 [graph_manager.cc:2937][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.641 [graph_manager.cc:2943][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.657 [graph_manager.cc:2950][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.831 [graph_manager.cc:2958][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [40] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.864 [graph_manager.cc:1132][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [717] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.036.973 [graph_manager.cc:1135][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [95] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.037.012 [graph_manager.cc:2975][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.037.146 [graph_manager.cc:2981][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [120] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.037.163 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.037.173 [graph_manager.cc:2986][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.037.183 [graph_manager.cc:1136][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [193] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.037.303 [graph_manager.cc:3555][EVENT]167130 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [92] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.037.371 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.037.388 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.037.526 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [119] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.037.560 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.037.601 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [29] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.037.624 [graph_builder.cc:865][EVENT]167130 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [291] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.037.764 [graph_builder.cc:288][EVENT]167130 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [123] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.037.873 [graph_builder.cc:293][EVENT]167130 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [93] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.038.070 [model_builder.cc:1133][EVENT]167130 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [106] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.038.336 [block_mem_assigner.cc:4069][EVENT]170657 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164040,python):2024-01-10-11:39:29.038.349 [block_mem_assigner.cc:4069][EVENT]170658 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164040,python):2024-01-10-11:39:29.038.721 [graph_mem_assigner.cc:2166][EVENT]167130 SetInputOffset:[IMAS]AfterAssignMemory : online_22 memoffset[1024], memtype[2], theory_min[2048], zero_copy[1024], total_size[1024], no_reuse[1024], streams[1], topo_mode[DFS], mop[], io_reuse[0:0], alloc_mode[] [INFO] GE(164040,python):2024-01-10-11:39:29.038.820 [model_builder.cc:1144][EVENT]167130 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [727] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.038.848 [model_builder.cc:1152][EVENT]167130 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.038.863 [model_builder.cc:1157][EVENT]167130 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.038.990 [model_builder.cc:1167][EVENT]167130 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [115] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.039.009 [model_builder.cc:1174][EVENT]167130 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.039.032 [model_builder.cc:1180][EVENT]167130 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.039.068 [model_builder.cc:1184][EVENT]167130 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [25] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.039.089 [graph_builder.cc:304][EVENT]167130 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [1194] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:39:29.039.190 [logger.cc:1071] 167130 ModelBindStream: model_id=832, stream_id=65, flag=0. [INFO] GE(164040,python):2024-01-10-11:39:29.039.296 [task_generator.cc:804][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.039.366 [task_generator.cc:805][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [55] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.040.176 [task_generator.cc:814][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [785] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.040.192 [task_generator.cc:954][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [902] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.040.256 [task_generator.cc:967][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [38] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:39:29.040.276 [logger.cc:1084] 167130 ModelUnbindStream: model_id=832, stream_id=65, [INFO] GE(164040,python):2024-01-10-11:39:29.040.334 [graph_builder.cc:310][EVENT]167130 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [1232] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.040.460 [graph_manager.cc:1152][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3255] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.040.478 [graph_manager.cc:1164][EVENT]167130 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:39:29.040.510 [graph_manager.cc:1271][EVENT]167130 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [198705] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.040.522 [graph_manager.cc:1272][EVENT]167130 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:39:29.040.837 [atrace_api.c:93](tid:167130) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:39:29.040.853 [atrace_api.c:95](tid:167130) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:39:29.041.523 [model_introduction.cc:236][EVENT]167130 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(164040,python):2024-01-10-11:39:29.041.546 [model_introduction.cc:294][EVENT]167130 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(164040,python):2024-01-10-11:39:29.041.560 [model_introduction.cc:366][EVENT]167130 Data:model io_info size:222 [INFO] GE(164040,python):2024-01-10-11:39:29.112.053 [graph_var_manager.cc:1424][EVENT]167129 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:39:29.112.157 [graph_manager.cc:1248][EVENT]167129 PreRun:PreRun start: graph node size 5, session id 24, graph id 23, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:39:29.112.574 [atrace_api.c:28](tid:167129) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:39:29.112.629 [trace_rb_log.c:84](tid:167129) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:39:29.112.642 [atrace_api.c:32](tid:167129) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:39:29.112.659 [client_manager.cpp:157][SetProfilingCallback][tid:167129] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:39:29.113.140 [parallel_partitioner.cc:165][EVENT]167129 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.113.179 [parallel_partitioner.cc:178][EVENT]167129 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.113.227 [graph_prepare.cc:1378][EVENT]167129 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.113.399 [graph_manager.cc:1050][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [190] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.113.424 [graph_manager.cc:1052][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.113.624 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.113.654 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.113.779 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [113] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.113.794 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.113.843 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.113.856 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.113.872 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.113.980 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.114.002 [graph_manager.cc:1054][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [534] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.114.218 [graph_manager.cc:1055][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [203] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.115.565 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.115.597 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [6] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.115.609 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.115.618 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [461] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.115.627 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.115.636 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.115.644 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.115.653 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.115.662 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.119.925 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.119.957 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.119.968 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.119.989 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [361] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.119.999 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.120.008 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.120.017 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.120.026 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.120.034 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.121.412 [graph_manager.cc:1056][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [7173] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.121.483 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.121.502 [graph_prepare.cc:1982][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [54] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.122.154 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.122.183 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.194 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.204 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [384] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.213 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.221 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.122.230 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.239 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.247 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.298 [graph_prepare.cc:1983][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [783] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.122.324 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.122.336 [graph_prepare.cc:1984][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.122.350 [graph_prepare.cc:1985][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.122.364 [graph_prepare.cc:1986][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.122.386 [graph_prepare.cc:1987][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.122.402 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.122.413 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.122.428 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.122.531 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.543 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.551 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.560 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.568 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.577 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.586 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.594 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.603 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of StopGradientPass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.611 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.619 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [4] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.627 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.635 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [4] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.643 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.652 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.660 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.122.683 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.122.697 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.122.732 [graph_prepare.cc:1988][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [336] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.122.752 [graph_manager.cc:1065][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1305] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.135.418 [graph_manager.cc:1077][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12646] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.135.529 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.135.583 [graph_manager.cc:1080][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [125] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.141.229 [graph_manager.cc:1081][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [5630] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.141.274 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.141.291 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.141.303 [graph_manager.cc:1082][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.141.335 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.141.351 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.141.367 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.141.542 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [164] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.141.560 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.141.666 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [95] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.141.682 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.141.765 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [44] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.141.787 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.141.817 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.141.916 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [87] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.141.935 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.141.949 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.141.971 [graph_manager.cc:2700][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [641] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.142.238 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.142.253 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.142.264 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.142.273 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [4] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.142.282 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.142.291 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CastRemovePass is [45] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.142.299 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [5] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.142.307 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.142.316 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [11] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.142.324 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [5] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.142.333 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [19] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.142.341 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.142.349 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [25] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.142.358 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [10] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.142.366 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.142.376 [graph_manager.cc:2741][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [386] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.142.385 [graph_manager.cc:2752][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.142.408 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.142.422 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.142.445 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.142.462 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.142.475 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.142.498 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.142.522 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.142.537 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.142.551 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.142.562 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.142.576 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.142.588 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.142.614 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.142.628 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.142.637 [graph_manager.cc:2810][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [233] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.142.687 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.142.699 [graph_manager.cc:2821][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [53] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.142.727 [graph_manager.cc:1087][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1405] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.143.316 [graph_manager.cc:1088][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [575] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.143.384 [graph_manager.cc:1089][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.143.408 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.143.426 [graph_manager.cc:1097][EVENT]167129 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:39:29.143.449 [graph_manager.cc:3325][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.153.540 [engine_place.cc:144][EVENT]167129 Run:The time cost of AIcoreEngine::CheckSupported is [9838] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.153.573 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.153.583 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.153.702 [graph_manager.cc:3351][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10223] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.153.737 [graph_manager.cc:3364][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.153.822 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.153.854 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.154.061 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [196] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.154.110 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.154.161 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.154.201 [graph_manager.cc:3405][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [451] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.154.220 [graph_manager.cc:3412][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.442.621 [graph_manager.cc:3422][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [288384] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.442.705 [graph_manager.cc:3428][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.442.967 [graph_manager.cc:3467][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [232] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.442.991 [graph_manager.cc:3377][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [289242] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.010 [graph_manager.cc:1106][EVENT]167129 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [299569] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.022 [graph_manager.cc:1115][EVENT]167129 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:39:29.443.052 [graph_manager.cc:1130][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.089 [graph_manager.cc:1131][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [24] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.124 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.147 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.157 [graph_manager.cc:2837][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [51] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.347 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [40] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.443.384 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.443.394 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [7] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.443.403 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.443.412 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [12] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.443.421 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [22] micro second, call num is [11] [INFO] GE(164040,python):2024-01-10-11:39:29.443.432 [graph_manager.cc:2864][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [254] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.445 [graph_manager.cc:2872][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.468 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.484 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.503 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.519 [compile_nodes_pass.cc:88][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.530 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.540 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.669 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [120] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.726 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [44] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.742 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.756 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.773 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.783 [graph_manager.cc:2927][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [320] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.795 [graph_manager.cc:2937][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.812 [graph_manager.cc:2943][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.443.831 [graph_manager.cc:2950][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.444.053 [graph_manager.cc:2958][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [66] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.444.090 [graph_manager.cc:1132][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [987] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.444.201 [graph_manager.cc:1135][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [97] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.444.244 [graph_manager.cc:2975][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [24] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.444.283 [graph_manager.cc:2981][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.444.298 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.444.309 [graph_manager.cc:2986][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.444.318 [graph_manager.cc:1136][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [98] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.444.706 [graph_manager.cc:3555][EVENT]167129 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [342] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.444.861 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [41] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.444.898 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.445.092 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [183] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.445.134 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.445.186 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.445.216 [graph_builder.cc:865][EVENT]167129 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [431] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:39:29.445.684 [logger.cc:1071] 167129 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:39:29.445.766 [task_generator.cc:804][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [148] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.445.866 [task_generator.cc:805][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [87] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.447.772 [task_generator.cc:814][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1889] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.447.788 [task_generator.cc:954][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2170] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.447.879 [task_generator.cc:967][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [43] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:39:29.447.905 [logger.cc:1084] 167129 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:39:29.497.414 [graph_manager.cc:1152][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [53058] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.497.510 [graph_manager.cc:1164][EVENT]167129 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:39:29.497.563 [graph_manager.cc:1271][EVENT]167129 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [384518] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.497.576 [graph_manager.cc:1272][EVENT]167129 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:39:29.497.956 [atrace_api.c:93](tid:167129) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:39:29.497.977 [atrace_api.c:95](tid:167129) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:39:29.691.712 [graph_converter.cc:838][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [66530] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.692.043 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [232] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.693.964 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [1890] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.694.470 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [469] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.694.500 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [501] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.694.793 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [279] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.694.888 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [67] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.694.968 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [59] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.695.551 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [563] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.695.798 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [217] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.695.822 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [244] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.695.897 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [63] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.695.965 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [54] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.696.035 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [56] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.696.315 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [265] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.696.546 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [208] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.696.566 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [230] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.696.638 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [62] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.696.737 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [56] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.696.760 [graph_converter.cc:849][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4956] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.697.662 [graph_converter.cc:853][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [890] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.700.153 [graph_converter.cc:857][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2435] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.700.648 [graph_converter.cc:862][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [461] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.831.239 [graph_var_manager.cc:1424][EVENT]167130 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:39:29.831.342 [graph_manager.cc:1248][EVENT]167130 PreRun:PreRun start: graph node size 5, session id 25, graph id 24, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:39:29.831.588 [atrace_api.c:28](tid:167130) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:39:29.831.619 [trace_rb_log.c:84](tid:167130) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:39:29.831.632 [atrace_api.c:32](tid:167130) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:39:29.831.649 [client_manager.cpp:157][SetProfilingCallback][tid:167130] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:39:29.832.019 [parallel_partitioner.cc:165][EVENT]167130 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.832.058 [parallel_partitioner.cc:178][EVENT]167130 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.832.105 [graph_prepare.cc:1378][EVENT]167130 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.832.280 [graph_manager.cc:1050][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [192] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.832.304 [graph_manager.cc:1052][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.832.469 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.832.501 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.832.549 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.832.562 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.832.607 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.832.621 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.832.667 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.832.772 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.832.793 [graph_manager.cc:1054][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [475] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.833.010 [graph_manager.cc:1055][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [203] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.834.418 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.834.450 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.834.462 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.834.472 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [468] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.834.481 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.834.489 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.834.498 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.834.506 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.834.514 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [7] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.838.982 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.839.016 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.839.027 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.839.037 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [329] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.839.046 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.839.055 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.839.064 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.839.073 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.839.081 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.840.436 [graph_manager.cc:1056][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [7405] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.840.521 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.840.541 [graph_prepare.cc:1982][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [58] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.841.142 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.841.170 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.182 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.192 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [332] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.201 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.209 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.841.218 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.227 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.235 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.288 [graph_prepare.cc:1983][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [732] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.841.315 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.841.328 [graph_prepare.cc:1984][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.841.343 [graph_prepare.cc:1985][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.841.358 [graph_prepare.cc:1986][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.841.369 [graph_prepare.cc:1987][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.841.383 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.841.395 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.841.410 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.841.513 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.526 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.546 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.555 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.564 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DropOutPass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.572 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.581 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.589 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.597 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.606 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.614 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.623 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.631 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.639 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.647 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.655 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:29.841.679 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.841.732 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.841.768 [graph_prepare.cc:1988][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [390] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.841.780 [graph_manager.cc:1065][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1298] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.854.432 [graph_manager.cc:1077][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12631] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.854.512 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.854.565 [graph_manager.cc:1080][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [91] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.859.964 [graph_manager.cc:1081][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [5381] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.860.020 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.860.038 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.860.049 [graph_manager.cc:1082][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.860.082 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.860.098 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.860.113 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.860.270 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [146] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.860.287 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.860.380 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [83] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.860.396 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.860.449 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [41] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.860.470 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.860.500 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.860.591 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [80] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.860.609 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.860.622 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.860.632 [graph_manager.cc:2700][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [555] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.860.883 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.860.899 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AddNPass is [3] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.860.909 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.860.918 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.860.927 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.860.945 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CastRemovePass is [43] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.860.955 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.860.964 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.860.973 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [12] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.860.981 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.860.990 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [19] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.860.998 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.861.006 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [21] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.861.015 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [11] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.861.023 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.861.032 [graph_manager.cc:2741][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [382] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.861.041 [graph_manager.cc:2752][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.861.066 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.861.079 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.861.102 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.861.118 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.861.130 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.861.144 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.861.166 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.861.182 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.861.196 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.861.206 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.861.219 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.861.239 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.861.262 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.861.276 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.861.285 [graph_manager.cc:2810][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [224] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.861.333 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.861.346 [graph_manager.cc:2821][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [53] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.861.374 [graph_manager.cc:1087][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1305] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.862.009 [graph_manager.cc:1088][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [620] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.862.076 [graph_manager.cc:1089][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [33] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.862.099 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.862.118 [graph_manager.cc:1097][EVENT]167130 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:39:29.862.141 [graph_manager.cc:3325][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.870.033 [engine_place.cc:144][EVENT]167130 Run:The time cost of AIcoreEngine::CheckSupported is [7669] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.870.067 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.870.078 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.870.182 [graph_manager.cc:3351][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [8028] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.870.204 [graph_manager.cc:3364][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.870.288 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.870.321 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.870.522 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [189] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.870.570 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [33] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.870.634 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [40] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.870.673 [graph_manager.cc:3405][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [456] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.870.691 [graph_manager.cc:3412][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.977.858 [graph_manager.cc:3422][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [107151] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.977.906 [graph_manager.cc:3428][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.088 [graph_manager.cc:3467][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [158] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.110 [graph_manager.cc:3377][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [107893] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.128 [graph_manager.cc:1106][EVENT]167130 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [115996] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.141 [graph_manager.cc:1115][EVENT]167130 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:39:29.978.167 [graph_manager.cc:1130][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.199 [graph_manager.cc:1131][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.227 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.249 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.259 [graph_manager.cc:2837][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [44] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.403 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [29] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.978.418 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.978.427 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.978.436 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of BitcastPass is [3] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.978.445 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [9] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.978.454 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [15] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:29.978.464 [graph_manager.cc:2864][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [187] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.492 [graph_manager.cc:2872][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.513 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.529 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.547 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.563 [compile_nodes_pass.cc:88][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.573 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.583 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.702 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [109] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.752 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.766 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.781 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.796 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.805 [graph_manager.cc:2927][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [296] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.818 [graph_manager.cc:2937][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.833 [graph_manager.cc:2943][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.978.844 [graph_manager.cc:2950][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.979.043 [graph_manager.cc:2958][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [60] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.979.077 [graph_manager.cc:1132][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [865] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.979.158 [graph_manager.cc:1135][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [67] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.979.197 [graph_manager.cc:2975][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.979.231 [graph_manager.cc:2981][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.979.265 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.979.275 [graph_manager.cc:2986][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.979.284 [graph_manager.cc:1136][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [109] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.979.627 [graph_manager.cc:3555][EVENT]167130 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [300] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.979.768 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.979.800 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.979.976 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [164] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.980.014 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [25] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.980.059 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [34] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.980.085 [graph_builder.cc:865][EVENT]167130 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [384] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:39:29.980.496 [logger.cc:1071] 167130 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:39:29.980.532 [task_generator.cc:804][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [84] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.980.622 [task_generator.cc:805][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [78] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.982.510 [task_generator.cc:814][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1872] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.982.528 [task_generator.cc:954][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2081] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.982.602 [task_generator.cc:967][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [39] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:39:29.982.628 [logger.cc:1084] 167130 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:39:29.983.668 [graph_manager.cc:1152][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4347] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.983.705 [graph_manager.cc:1164][EVENT]167130 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:39:29.983.743 [graph_manager.cc:1271][EVENT]167130 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [151815] micro second. [INFO] GE(164040,python):2024-01-10-11:39:29.983.756 [graph_manager.cc:1272][EVENT]167130 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:39:29.984.082 [atrace_api.c:93](tid:167130) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:39:29.984.111 [atrace_api.c:95](tid:167130) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:39:30.009.996 [graph_converter.cc:838][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [10218] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.010.231 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [177] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.011.908 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [1645] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.012.381 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [440] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.012.411 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [473] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.012.682 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [256] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.012.772 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [64] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.012.847 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [53] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.013.379 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [515] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.013.610 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [202] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.013.637 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [230] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.013.716 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [67] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.013.783 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [50] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.013.847 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [50] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.014.100 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [238] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.014.309 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [189] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.014.329 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [211] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.014.396 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [56] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.014.459 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [49] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.014.478 [graph_converter.cc:849][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4431] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.015.257 [graph_converter.cc:853][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [768] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.017.374 [graph_converter.cc:857][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2087] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.017.829 [graph_converter.cc:862][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [422] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.103.344 [graph_var_manager.cc:1424][EVENT]167131 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:39:30.103.445 [graph_manager.cc:1248][EVENT]167131 PreRun:PreRun start: graph node size 7, session id 26, graph id 25, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:39:30.104.097 [atrace_api.c:28](tid:167131) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:39:30.104.154 [trace_rb_log.c:84](tid:167131) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:39:30.104.167 [atrace_api.c:32](tid:167131) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:39:30.104.184 [client_manager.cpp:157][SetProfilingCallback][tid:167131] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:39:30.104.907 [parallel_partitioner.cc:165][EVENT]167131 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.104.951 [parallel_partitioner.cc:178][EVENT]167131 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.105.001 [graph_prepare.cc:1378][EVENT]167131 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.105.397 [graph_manager.cc:1050][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [416] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.105.426 [graph_manager.cc:1052][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.105.611 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.105.644 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.105.736 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [81] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.105.752 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.105.798 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.105.811 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.105.828 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.105.942 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.105.963 [graph_manager.cc:1054][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [525] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.106.183 [graph_manager.cc:1055][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [206] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.107.669 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of AssertPass is [5] micro second, call num is [14] [INFO] GE(164040,python):2024-01-10-11:39:30.107.700 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.107.735 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.107.746 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of InferShapePass is [526] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.107.755 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [20] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.107.764 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [5] micro second, call num is [14] [INFO] GE(164040,python):2024-01-10-11:39:30.107.772 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.107.781 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [25] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.107.789 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of InferValuePass is [7] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.109.409 [graph_manager.cc:1056][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3206] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.109.487 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.109.508 [graph_prepare.cc:1982][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [64] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.110.195 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [14] [INFO] GE(164040,python):2024-01-10-11:39:30.110.224 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [6] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.235 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.244 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of InferShapePass is [433] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.253 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.261 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [14] [INFO] GE(164040,python):2024-01-10-11:39:30.110.270 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.278 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.287 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.315 [graph_prepare.cc:1983][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [793] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.110.340 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.110.351 [graph_prepare.cc:1984][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.110.365 [graph_prepare.cc:1985][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.110.390 [graph_prepare.cc:1986][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.110.401 [graph_prepare.cc:1987][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.110.417 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.110.428 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.110.442 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.110.566 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.579 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.589 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrintOpPass is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.597 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.606 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of DropOutPass is [0] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.614 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.623 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.631 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.640 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.648 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [5] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.656 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.665 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SnapshotPass is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.673 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.681 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [8] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.689 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.697 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.110.723 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.110.745 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.110.784 [graph_prepare.cc:1988][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [374] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.110.798 [graph_manager.cc:1065][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1354] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.123.649 [graph_manager.cc:1077][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12830] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.123.733 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.123.787 [graph_manager.cc:1080][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [99] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.089 [graph_manager.cc:1081][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [4286] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.131 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.148 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.160 [graph_manager.cc:1082][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.194 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.209 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.223 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.258 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [24] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.271 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.288 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.301 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.348 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.369 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.399 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.432 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [24] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.448 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.471 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.480 [graph_manager.cc:2700][EVENT]167131 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [292] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.639 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of EnterPass is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.128.654 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.128.663 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.128.672 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.128.680 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.128.689 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of CastRemovePass is [13] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.128.697 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.128.705 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.128.713 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.128.722 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.128.730 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.128.738 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.128.746 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.128.755 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.128.763 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.128.772 [graph_manager.cc:2741][EVENT]167131 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [274] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.781 [graph_manager.cc:2752][EVENT]167131 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.804 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.816 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.836 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.861 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.874 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.887 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.908 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.923 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.937 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.947 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.960 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.973 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.128.994 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.129.008 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.129.018 [graph_manager.cc:2810][EVENT]167131 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [218] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.129.056 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.129.067 [graph_manager.cc:2821][EVENT]167131 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [40] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.129.096 [graph_manager.cc:1087][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [916] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.129.231 [graph_manager.cc:1088][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [123] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.129.278 [graph_manager.cc:1089][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [25] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.129.298 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.129.338 [graph_manager.cc:1097][EVENT]167131 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:39:30.129.361 [graph_manager.cc:3325][EVENT]167131 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.129.884 [engine_place.cc:144][EVENT]167131 Run:The time cost of AIcoreEngine::CheckSupported is [406] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.129.914 [engine_place.cc:144][EVENT]167131 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.129.924 [engine_place.cc:144][EVENT]167131 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.130.025 [graph_manager.cc:3351][EVENT]167131 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [651] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.130.045 [graph_manager.cc:3364][EVENT]167131 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.130.115 [engine_partitioner.cc:1139][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [24] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.130.133 [engine_partitioner.cc:1142][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.130.374 [engine_partitioner.cc:1148][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [230] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.130.426 [engine_partitioner.cc:1155][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.130.480 [engine_partitioner.cc:1164][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [42] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.130.517 [graph_manager.cc:3405][EVENT]167131 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [459] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.130.534 [graph_manager.cc:3412][EVENT]167131 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.142.580 [graph_manager.cc:3422][EVENT]167131 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [12033] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.142.616 [graph_manager.cc:3428][EVENT]167131 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.142.765 [graph_manager.cc:3467][EVENT]167131 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [128] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.142.784 [graph_manager.cc:3377][EVENT]167131 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [12727] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.142.800 [graph_manager.cc:1106][EVENT]167131 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [13445] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.142.813 [graph_manager.cc:1115][EVENT]167131 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:39:30.142.835 [graph_manager.cc:1130][EVENT]167131 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.142.865 [graph_manager.cc:1131][EVENT]167131 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.142.890 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.142.909 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.142.919 [graph_manager.cc:2837][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.026 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.143.040 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.143.049 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.143.058 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.143.066 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [8] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.143.075 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [6] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:39:30.143.084 [graph_manager.cc:2864][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [139] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.096 [graph_manager.cc:2872][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.116 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.130 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.147 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.161 [compile_nodes_pass.cc:88][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.171 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.180 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.291 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [102] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.321 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.335 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.349 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.363 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.372 [graph_manager.cc:2927][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [259] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.384 [graph_manager.cc:2937][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.407 [graph_manager.cc:2943][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.418 [graph_manager.cc:2950][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.599 [graph_manager.cc:2958][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [52] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.633 [graph_manager.cc:1132][EVENT]167131 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [754] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.704 [graph_manager.cc:1135][EVENT]167131 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [58] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.744 [graph_manager.cc:2975][EVENT]167131 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.775 [graph_manager.cc:2981][EVENT]167131 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.789 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.799 [graph_manager.cc:2986][EVENT]167131 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.808 [graph_manager.cc:1136][EVENT]167131 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [88] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.143.954 [graph_manager.cc:3555][EVENT]167131 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [109] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.144.061 [engine_partitioner.cc:1139][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.144.080 [engine_partitioner.cc:1142][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.144.270 [engine_partitioner.cc:1148][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [180] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.144.309 [engine_partitioner.cc:1155][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [25] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.144.351 [engine_partitioner.cc:1164][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.144.376 [graph_builder.cc:865][EVENT]167131 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [359] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:39:30.144.691 [logger.cc:1071] 167131 ModelBindStream: model_id=1344, stream_id=1601, flag=0. [INFO] GE(164040,python):2024-01-10-11:39:30.144.723 [task_generator.cc:804][EVENT]167131 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [71] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.144.796 [task_generator.cc:805][EVENT]167131 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [62] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.145.634 [task_generator.cc:814][EVENT]167131 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [823] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.145.658 [task_generator.cc:954][EVENT]167131 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1007] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.145.742 [task_generator.cc:967][EVENT]167131 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [58] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:39:30.145.763 [logger.cc:1084] 167131 ModelUnbindStream: model_id=1344, stream_id=1601, [INFO] GE(164040,python):2024-01-10-11:39:30.145.945 [graph_manager.cc:1152][EVENT]167131 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2108] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.145.964 [graph_manager.cc:1164][EVENT]167131 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:39:30.145.996 [graph_manager.cc:1271][EVENT]167131 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [41194] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.146.008 [graph_manager.cc:1272][EVENT]167131 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:39:30.146.320 [atrace_api.c:93](tid:167131) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:39:30.146.334 [atrace_api.c:95](tid:167131) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:39:30.153.473 [graph_converter.cc:838][EVENT]167131 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1718] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.153.559 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of ZeroCopy is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.154.217 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of CEM is [637] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.154.498 [copy_flow_launch_fuse.cc:395][EVENT]167131 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [251] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.154.521 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [277] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.154.576 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [43] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.154.612 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.154.646 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of ZeroCopy is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.154.857 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of CEM is [199] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.154.961 [copy_flow_launch_fuse.cc:395][EVENT]167131 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [86] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.154.976 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [102] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.155.014 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.155.045 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.155.077 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of ZeroCopy is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.155.186 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of CEM is [99] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.155.277 [copy_flow_launch_fuse.cc:395][EVENT]167131 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [79] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.155.301 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [102] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.155.338 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.155.367 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.155.381 [graph_converter.cc:849][EVENT]167131 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1865] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.155.683 [graph_converter.cc:853][EVENT]167131 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [291] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.156.713 [graph_converter.cc:857][EVENT]167131 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1012] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.156.928 [graph_converter.cc:862][EVENT]167131 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [186] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.227.203 [graph_var_manager.cc:1424][EVENT]167129 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:39:30.227.299 [graph_manager.cc:1248][EVENT]167129 PreRun:PreRun start: graph node size 5, session id 27, graph id 26, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:39:30.227.532 [atrace_api.c:28](tid:167129) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:39:30.227.563 [trace_rb_log.c:84](tid:167129) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:39:30.227.577 [atrace_api.c:32](tid:167129) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:39:30.227.594 [client_manager.cpp:157][SetProfilingCallback][tid:167129] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:39:30.227.949 [parallel_partitioner.cc:165][EVENT]167129 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.227.988 [parallel_partitioner.cc:178][EVENT]167129 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.228.035 [graph_prepare.cc:1378][EVENT]167129 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.228.182 [graph_manager.cc:1050][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [164] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.228.207 [graph_manager.cc:1052][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.228.360 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.228.391 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.228.441 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.228.454 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.228.501 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.228.544 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.228.561 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.228.666 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.228.688 [graph_manager.cc:1054][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [468] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.228.906 [graph_manager.cc:1055][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [204] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.230.149 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:30.230.180 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [6] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.230.192 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.230.201 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [427] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.230.210 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [15] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.230.219 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:30.230.227 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [10] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.230.236 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.230.244 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [7] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.232.608 [graph_manager.cc:1056][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3682] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.232.680 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.232.700 [graph_prepare.cc:1982][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [57] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.233.309 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:30.233.337 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.348 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.357 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [340] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.366 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.387 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [10] [INFO] GE(164040,python):2024-01-10-11:39:30.233.396 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.405 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.413 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.467 [graph_prepare.cc:1983][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [754] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.233.494 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.233.507 [graph_prepare.cc:1984][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.233.521 [graph_prepare.cc:1985][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.233.536 [graph_prepare.cc:1986][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.233.548 [graph_prepare.cc:1987][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.233.562 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.233.575 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.233.589 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.233.713 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.727 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.736 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.745 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.753 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.762 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [20] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.770 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.779 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.787 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.795 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.812 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.821 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.830 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.838 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.846 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.855 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [5] [INFO] GE(164040,python):2024-01-10-11:39:30.233.879 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.233.892 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.233.928 [graph_prepare.cc:1988][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [372] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.233.941 [graph_manager.cc:1065][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1299] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.246.060 [graph_manager.cc:1077][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12098] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.246.138 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.246.189 [graph_manager.cc:1080][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [92] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.255.074 [graph_manager.cc:1081][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [8869] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.255.123 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.255.140 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.255.152 [graph_manager.cc:1082][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.255.186 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.255.202 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.255.217 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.255.408 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [181] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.255.440 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.255.563 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [112] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.255.579 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.255.635 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [45] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.255.657 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.255.678 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.255.785 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [96] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.255.804 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.255.818 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.255.828 [graph_manager.cc:2700][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [648] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.256.123 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:30.256.140 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:30.256.150 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:30.256.160 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:30.256.168 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:30.256.178 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CastRemovePass is [53] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:30.256.186 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [6] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:30.256.195 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:30.256.203 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [12] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:30.256.211 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:30.256.220 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [19] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:30.256.228 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:30.256.236 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [23] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:30.256.254 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [14] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:30.256.263 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [5] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:30.256.273 [graph_manager.cc:2741][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [426] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.256.282 [graph_manager.cc:2752][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.256.307 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.256.319 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.256.344 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.256.360 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.256.372 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.256.385 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.256.408 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.256.423 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.256.436 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.256.446 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.256.458 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.256.471 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.256.497 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.256.510 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.256.519 [graph_manager.cc:2810][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [218] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.256.572 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [5] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:39:30.256.584 [graph_manager.cc:2821][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [56] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.256.612 [graph_manager.cc:1087][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1440] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.257.325 [graph_manager.cc:1088][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [691] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.257.395 [graph_manager.cc:1089][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.257.420 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.257.439 [graph_manager.cc:1097][EVENT]167129 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:39:30.257.461 [graph_manager.cc:3325][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.268.269 [engine_place.cc:144][EVENT]167129 Run:The time cost of AIcoreEngine::CheckSupported is [10532] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.268.301 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.268.312 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.268.416 [graph_manager.cc:3351][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10942] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.268.437 [graph_manager.cc:3364][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.268.529 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.268.565 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.268.778 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [201] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.268.829 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.268.879 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.268.918 [graph_manager.cc:3405][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [468] micro second. [INFO] GE(164040,python):2024-01-10-11:39:30.268.938 [graph_manager.cc:3412][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. / - \ | / - [INFO] GE(164040,python):2024-01-10-11:40:27.299.845 [graph_manager.cc:3422][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [57030891] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.299.940 [graph_manager.cc:3428][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.300.261 [graph_manager.cc:3467][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [290] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.300.285 [graph_manager.cc:3377][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [57031834] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.300.326 [graph_manager.cc:1106][EVENT]167129 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [57042872] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.300.340 [graph_manager.cc:1115][EVENT]167129 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:40:27.300.375 [graph_manager.cc:1130][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.300.415 [graph_manager.cc:1131][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [26] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.300.452 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.300.479 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.300.490 [graph_manager.cc:2837][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [57] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.300.695 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [43] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:40:27.300.710 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [5] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:40:27.300.719 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:40:27.300.728 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of BitcastPass is [3] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:40:27.300.737 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [13] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:40:27.300.745 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [20] micro second, call num is [12] [INFO] GE(164040,python):2024-01-10-11:40:27.300.757 [graph_manager.cc:2864][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [246] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.300.770 [graph_manager.cc:2872][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.300.793 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.300.810 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.300.829 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.300.844 [compile_nodes_pass.cc:88][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.300.854 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.300.865 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.301.015 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [131] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.301.118 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [89] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.301.134 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.301.149 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.301.166 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.301.176 [graph_manager.cc:2927][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [388] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.301.190 [graph_manager.cc:2937][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.301.207 [graph_manager.cc:2943][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.301.218 [graph_manager.cc:2950][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.301.567 [graph_manager.cc:2958][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [68] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.301.605 [graph_manager.cc:1132][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [1175] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.301.802 [graph_manager.cc:1135][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [183] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.301.854 [graph_manager.cc:2975][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.301.889 [graph_manager.cc:2981][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.301.904 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.301.915 [graph_manager.cc:2986][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.301.924 [graph_manager.cc:1136][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [104] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.302.345 [graph_manager.cc:3555][EVENT]167129 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [368] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.302.527 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [44] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.302.566 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.302.773 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [195] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.302.829 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.302.883 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [42] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.302.914 [graph_builder.cc:865][EVENT]167129 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [468] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:40:27.303.503 [logger.cc:1071] 167129 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:40:27.303.543 [task_generator.cc:804][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [163] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.303.658 [task_generator.cc:805][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [102] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.305.929 [task_generator.cc:814][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [2255] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.305.947 [task_generator.cc:954][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2568] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.306.028 [task_generator.cc:967][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [44] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:40:27.306.053 [logger.cc:1084] 167129 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:40:27.308.148 [graph_manager.cc:1152][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [6182] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.308.186 [graph_manager.cc:1164][EVENT]167129 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:40:27.308.235 [graph_manager.cc:1271][EVENT]167129 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [57080380] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.308.248 [graph_manager.cc:1272][EVENT]167129 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:40:27.308.664 [atrace_api.c:93](tid:167129) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:40:27.308.686 [atrace_api.c:95](tid:167129) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:40:27.339.380 [graph_converter.cc:838][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [11933] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.339.637 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [198] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.341.751 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [2085] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.342.287 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [503] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.342.314 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [533] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.342.623 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [295] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.342.725 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [76] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.342.809 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [63] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.343.427 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [585] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.343.700 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [244] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.343.725 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [270] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.343.806 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [69] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.343.881 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [59] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.343.956 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [61] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.344.242 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [271] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.344.485 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [223] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.344.507 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [246] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.344.584 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [66] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.344.656 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [58] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.344.677 [graph_converter.cc:849][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [5245] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.345.591 [graph_converter.cc:853][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [902] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.348.171 [graph_converter.cc:857][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2554] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.348.686 [graph_converter.cc:862][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [482] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.434.923 [graph_var_manager.cc:1424][EVENT]167129 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:40:27.435.033 [graph_manager.cc:1248][EVENT]167129 PreRun:PreRun start: graph node size 4, session id 28, graph id 27, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:40:27.435.277 [atrace_api.c:28](tid:167129) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:40:27.435.308 [trace_rb_log.c:84](tid:167129) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:40:27.435.321 [atrace_api.c:32](tid:167129) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:40:27.435.342 [client_manager.cpp:157][SetProfilingCallback][tid:167129] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:40:27.435.739 [parallel_partitioner.cc:165][EVENT]167129 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.435.778 [parallel_partitioner.cc:178][EVENT]167129 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.435.852 [graph_prepare.cc:1378][EVENT]167129 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.436.218 [graph_manager.cc:1050][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [384] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.436.246 [graph_manager.cc:1052][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.436.392 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.436.423 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.436.472 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.436.484 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.436.530 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.436.544 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.436.560 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.436.674 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.436.695 [graph_manager.cc:1054][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [436] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.436.927 [graph_manager.cc:1055][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [218] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.438.046 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:40:27.438.078 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.438.089 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.438.099 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [383] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.438.108 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.438.116 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:40:27.438.125 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.438.133 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.438.142 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.440.346 [graph_manager.cc:1056][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3397] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.440.415 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.440.435 [graph_prepare.cc:1982][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [56] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.440.964 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:40:27.440.992 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.004 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.013 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [282] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.022 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.030 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164040,python):2024-01-10-11:40:27.441.039 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.047 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.056 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.102 [graph_prepare.cc:1983][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [652] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.441.127 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.441.138 [graph_prepare.cc:1984][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.441.152 [graph_prepare.cc:1985][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.441.165 [graph_prepare.cc:1986][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.441.177 [graph_prepare.cc:1987][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.441.191 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.441.202 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.441.216 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.441.308 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.321 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.355 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.364 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [0] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.373 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.382 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.390 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [0] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.398 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.407 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.416 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.425 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.433 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.441 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.450 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.458 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.466 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.441.490 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.441.503 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.441.536 [graph_prepare.cc:1988][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [350] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.441.549 [graph_manager.cc:1065][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1170] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.455.338 [graph_manager.cc:1077][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13769] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.455.443 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.455.497 [graph_manager.cc:1080][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [121] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.464.869 [graph_manager.cc:1081][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [9356] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.464.930 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.464.947 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.464.960 [graph_manager.cc:1082][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.464.992 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.008 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.023 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.131 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [96] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.148 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.195 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.211 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.251 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [30] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.270 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.297 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.330 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.346 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.358 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.368 [graph_manager.cc:2700][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [381] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.498 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.465.513 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.465.522 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.465.531 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.465.540 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.465.558 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CastRemovePass is [9] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.465.567 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.465.576 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.465.584 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [4] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.465.592 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.465.601 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.465.609 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.465.617 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.465.626 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.465.634 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.465.644 [graph_manager.cc:2741][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [256] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.653 [graph_manager.cc:2752][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.676 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.698 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.719 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.734 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.747 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.761 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.783 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.800 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.813 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.824 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.837 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.859 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.879 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.892 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.901 [graph_manager.cc:2810][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [230] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.933 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.465.945 [graph_manager.cc:2821][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.465.974 [graph_manager.cc:1087][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [995] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.466.112 [graph_manager.cc:1088][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [125] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.466.154 [graph_manager.cc:1089][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.466.173 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.466.188 [graph_manager.cc:1097][EVENT]167129 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:40:27.466.210 [graph_manager.cc:3325][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.466.621 [engine_place.cc:144][EVENT]167129 Run:The time cost of AIcoreEngine::CheckSupported is [288] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.466.649 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.466.659 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.466.741 [graph_manager.cc:3351][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [518] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.466.760 [graph_manager.cc:3364][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.466.833 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.466.851 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.467.016 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [154] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.467.063 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.467.125 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.467.159 [graph_manager.cc:3405][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [387] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.467.177 [graph_manager.cc:3412][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.484.526 [graph_manager.cc:3422][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [17335] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.484.569 [graph_manager.cc:3428][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.484.716 [graph_manager.cc:3467][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [125] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.484.735 [graph_manager.cc:3377][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [17964] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.484.751 [graph_manager.cc:1106][EVENT]167129 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [18548] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.484.764 [graph_manager.cc:1115][EVENT]167129 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:40:27.484.787 [graph_manager.cc:1130][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.484.819 [graph_manager.cc:1131][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.484.844 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.484.862 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.484.872 [graph_manager.cc:2837][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.484.955 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.484.969 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.484.978 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.484.986 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.484.995 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.485.004 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [8] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:40:27.485.013 [graph_manager.cc:2864][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [125] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.037 [graph_manager.cc:2872][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.057 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.072 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.087 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.101 [compile_nodes_pass.cc:88][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.111 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.120 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.202 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [73] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.230 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.244 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.258 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.271 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.281 [graph_manager.cc:2927][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [226] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.293 [graph_manager.cc:2937][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.306 [graph_manager.cc:2943][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.317 [graph_manager.cc:2950][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.493 [graph_manager.cc:2958][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [40] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.526 [graph_manager.cc:1132][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [693] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.599 [graph_manager.cc:1135][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [59] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.631 [graph_manager.cc:2975][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.665 [graph_manager.cc:2981][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.727 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.740 [graph_manager.cc:2986][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.749 [graph_manager.cc:1136][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [135] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.896 [graph_manager.cc:3555][EVENT]167129 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [113] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.485.989 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.486.006 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.486.133 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [117] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.486.165 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.486.204 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.486.228 [graph_builder.cc:865][EVENT]167129 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [276] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:40:27.486.617 [logger.cc:1071] 167129 ModelBindStream: model_id=1856, stream_id=65, flag=0. [INFO] GE(164040,python):2024-01-10-11:40:27.486.652 [task_generator.cc:804][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [134] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.486.716 [task_generator.cc:805][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [52] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.487.459 [task_generator.cc:814][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [727] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.487.473 [task_generator.cc:954][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [957] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.487.533 [task_generator.cc:967][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [33] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:40:27.487.551 [logger.cc:1084] 167129 ModelUnbindStream: model_id=1856, stream_id=65, [INFO] GE(164040,python):2024-01-10-11:40:27.488.555 [graph_manager.cc:1152][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2778] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.488.588 [graph_manager.cc:1164][EVENT]167129 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:40:27.488.623 [graph_manager.cc:1271][EVENT]167129 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [52981] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.488.636 [graph_manager.cc:1272][EVENT]167129 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:40:27.488.950 [atrace_api.c:93](tid:167129) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:40:27.488.977 [atrace_api.c:95](tid:167129) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:40:27.500.815 [graph_converter.cc:838][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [3890] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.500.989 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [124] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.501.490 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [478] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.501.738 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [221] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.501.762 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [246] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.501.985 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [211] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.502.027 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.502.059 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.502.263 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [192] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.502.349 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [66] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.502.364 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [81] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.502.393 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.502.419 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.502.446 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.502.525 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [70] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.502.594 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [58] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.502.605 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [69] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.502.631 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.502.656 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.502.669 [graph_converter.cc:849][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1810] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.502.896 [graph_converter.cc:853][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [217] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.503.628 [graph_converter.cc:857][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [715] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.503.783 [graph_converter.cc:862][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [127] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.582.312 [graph_var_manager.cc:1424][EVENT]167130 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:40:27.582.416 [graph_manager.cc:1248][EVENT]167130 PreRun:PreRun start: graph node size 7, session id 29, graph id 28, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:40:27.582.873 [atrace_api.c:28](tid:167130) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:40:27.582.929 [trace_rb_log.c:84](tid:167130) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:40:27.582.942 [atrace_api.c:32](tid:167130) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:40:27.582.959 [client_manager.cpp:157][SetProfilingCallback][tid:167130] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:40:27.583.467 [parallel_partitioner.cc:165][EVENT]167130 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.583.508 [parallel_partitioner.cc:178][EVENT]167130 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.583.559 [graph_prepare.cc:1378][EVENT]167130 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.583.729 [graph_manager.cc:1050][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [191] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.583.754 [graph_manager.cc:1052][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.583.938 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.583.969 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.584.019 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.584.033 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.584.079 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.584.093 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.584.110 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.584.222 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.584.243 [graph_manager.cc:1054][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [476] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.584.458 [graph_manager.cc:1055][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [202] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.586.005 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [14] [INFO] GE(164040,python):2024-01-10-11:40:27.586.037 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.586.078 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [6] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.586.089 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [573] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.586.098 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [20] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.586.107 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [14] [INFO] GE(164040,python):2024-01-10-11:40:27.586.115 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.586.124 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [22] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.586.132 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [7] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.587.777 [graph_manager.cc:1056][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3297] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.587.854 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [6] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.587.873 [graph_prepare.cc:1982][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [62] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.588.576 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [14] [INFO] GE(164040,python):2024-01-10-11:40:27.588.604 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [0] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.588.615 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.588.625 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferShapePass is [435] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.588.634 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [15] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.588.643 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [14] [INFO] GE(164040,python):2024-01-10-11:40:27.588.652 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [9] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.588.660 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.588.668 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.588.696 [graph_prepare.cc:1983][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [810] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.588.722 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.588.733 [graph_prepare.cc:1984][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.588.748 [graph_prepare.cc:1985][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.588.774 [graph_prepare.cc:1986][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.588.786 [graph_prepare.cc:1987][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.588.802 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.588.814 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.588.828 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.588.951 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.588.965 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondPass is [5] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.588.974 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.588.983 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.588.991 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.589.000 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.589.008 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.589.017 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.589.025 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of StopGradientPass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.589.033 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.589.041 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.589.049 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SnapshotPass is [0] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.589.057 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.589.065 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [9] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.589.073 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.589.081 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [5] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.589.106 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.589.127 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.589.166 [graph_prepare.cc:1988][EVENT]167130 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [371] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.589.180 [graph_manager.cc:1065][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1369] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.602.130 [graph_manager.cc:1077][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12931] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.602.213 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.602.266 [graph_manager.cc:1080][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [99] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.606.561 [graph_manager.cc:1081][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [4278] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.606.604 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.606.620 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.606.632 [graph_manager.cc:1082][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.606.665 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.606.679 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.606.694 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.606.731 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.606.745 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.606.762 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.606.776 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.606.827 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [40] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.606.848 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.606.878 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.606.913 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [24] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.606.929 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.606.953 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.606.964 [graph_manager.cc:2700][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [305] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.125 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.607.139 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.607.149 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.607.158 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.607.167 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.607.176 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CastRemovePass is [13] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.607.184 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.607.193 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.607.201 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.607.209 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.607.218 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.607.226 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.607.234 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.607.242 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [6] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.607.251 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.607.260 [graph_manager.cc:2741][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [277] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.269 [graph_manager.cc:2752][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.293 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.304 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.325 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.340 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.360 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.373 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.394 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.409 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.422 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.432 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.445 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.457 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.480 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.493 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.502 [graph_manager.cc:2810][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [214] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.542 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.607.555 [graph_manager.cc:2821][EVENT]167130 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [44] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.583 [graph_manager.cc:1087][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [932] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.720 [graph_manager.cc:1088][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [124] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.769 [graph_manager.cc:1089][EVENT]167130 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.789 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.607.831 [graph_manager.cc:1097][EVENT]167130 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:40:27.607.854 [graph_manager.cc:3325][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.608.373 [engine_place.cc:144][EVENT]167130 Run:The time cost of AIcoreEngine::CheckSupported is [400] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.608.402 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.608.412 [engine_place.cc:144][EVENT]167130 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.608.517 [graph_manager.cc:3351][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [650] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.608.537 [graph_manager.cc:3364][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.608.610 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [25] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.608.629 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.608.874 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [234] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.608.926 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.608.978 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [40] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.609.018 [graph_manager.cc:3405][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [468] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.609.037 [graph_manager.cc:3412][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.103 [graph_manager.cc:3422][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [12051] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.139 [graph_manager.cc:3428][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.293 [graph_manager.cc:3467][EVENT]167130 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [134] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.312 [graph_manager.cc:3377][EVENT]167130 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [12764] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.329 [graph_manager.cc:1106][EVENT]167130 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [13482] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.342 [graph_manager.cc:1115][EVENT]167130 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:40:27.621.365 [graph_manager.cc:1130][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.395 [graph_manager.cc:1131][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.420 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.437 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.447 [graph_manager.cc:2837][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [36] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.555 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.621.569 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.621.578 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.621.587 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of BitcastPass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.621.596 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.621.604 [base_pass.cc:339][EVENT]167130 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.621.614 [graph_manager.cc:2864][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [140] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.626 [graph_manager.cc:2872][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.644 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.659 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.675 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.716 [compile_nodes_pass.cc:88][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.741 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [57] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.751 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.864 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [103] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.894 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.908 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.922 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.936 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.945 [graph_manager.cc:2927][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [304] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.957 [graph_manager.cc:2937][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.981 [graph_manager.cc:2943][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.621.992 [graph_manager.cc:2950][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.622.170 [graph_manager.cc:2958][EVENT]167130 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [52] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.622.205 [graph_manager.cc:1132][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [796] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.622.277 [graph_manager.cc:1135][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [59] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.622.321 [graph_manager.cc:2975][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.622.356 [graph_manager.cc:2981][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.622.371 [pass_manager.cc:82][EVENT]167130 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.622.381 [graph_manager.cc:2986][EVENT]167130 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.622.390 [graph_manager.cc:1136][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [96] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.622.540 [graph_manager.cc:3555][EVENT]167130 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [113] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.622.651 [engine_partitioner.cc:1139][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [24] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.622.669 [engine_partitioner.cc:1142][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.622.861 [engine_partitioner.cc:1148][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [182] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.622.901 [engine_partitioner.cc:1155][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [26] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.622.944 [engine_partitioner.cc:1164][EVENT]167130 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.622.969 [graph_builder.cc:865][EVENT]167130 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [365] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:40:27.623.292 [logger.cc:1071] 167130 ModelBindStream: model_id=64, stream_id=321, flag=0. [INFO] GE(164040,python):2024-01-10-11:40:27.623.324 [task_generator.cc:804][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [73] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.623.400 [task_generator.cc:805][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [64] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.624.204 [task_generator.cc:814][EVENT]167130 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [789] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.624.229 [task_generator.cc:954][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [978] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.624.288 [task_generator.cc:967][EVENT]167130 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [33] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:40:27.624.307 [logger.cc:1084] 167130 ModelUnbindStream: model_id=64, stream_id=321, [INFO] GE(164040,python):2024-01-10-11:40:27.624.491 [graph_manager.cc:1152][EVENT]167130 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2070] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.624.509 [graph_manager.cc:1164][EVENT]167130 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:40:27.624.541 [graph_manager.cc:1271][EVENT]167130 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [41179] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.624.551 [graph_manager.cc:1272][EVENT]167130 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:40:27.624.875 [atrace_api.c:93](tid:167130) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:40:27.624.890 [atrace_api.c:95](tid:167130) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:40:27.632.319 [graph_converter.cc:838][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1762] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.632.404 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.633.052 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [627] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.633.334 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [254] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.633.357 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [280] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.633.413 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [44] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.633.449 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.633.483 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.633.737 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [242] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.633.847 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [88] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.633.863 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [104] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.633.901 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.633.932 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.633.964 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.634.078 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CEM is [103] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.634.170 [copy_flow_launch_fuse.cc:395][EVENT]167130 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [79] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.634.193 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [104] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.634.231 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.634.261 [base_optimizer.cc:70][EVENT]167130 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.634.276 [graph_converter.cc:849][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1916] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.634.584 [graph_converter.cc:853][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [298] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.635.616 [graph_converter.cc:857][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1014] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.635.836 [graph_converter.cc:862][EVENT]167130 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [191] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.702.839 [graph_var_manager.cc:1424][EVENT]167129 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:40:27.702.944 [graph_manager.cc:1248][EVENT]167129 PreRun:PreRun start: graph node size 7, session id 30, graph id 29, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:40:27.703.185 [atrace_api.c:28](tid:167129) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:40:27.703.216 [trace_rb_log.c:84](tid:167129) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:40:27.703.229 [atrace_api.c:32](tid:167129) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:40:27.703.246 [client_manager.cpp:157][SetProfilingCallback][tid:167129] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:40:27.703.630 [parallel_partitioner.cc:165][EVENT]167129 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.703.670 [parallel_partitioner.cc:178][EVENT]167129 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.703.720 [graph_prepare.cc:1378][EVENT]167129 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.703.867 [graph_manager.cc:1050][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [167] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.703.892 [graph_manager.cc:1052][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.704.075 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.704.107 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.704.157 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.704.170 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.704.217 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.704.257 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.704.274 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.704.383 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.704.404 [graph_manager.cc:1054][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [498] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.704.620 [graph_manager.cc:1055][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [203] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.706.116 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [14] [INFO] GE(164040,python):2024-01-10-11:40:27.706.147 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.706.159 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.706.168 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [564] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.706.177 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [17] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.706.186 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [14] [INFO] GE(164040,python):2024-01-10-11:40:27.706.195 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.706.203 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [22] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.706.211 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.707.840 [graph_manager.cc:1056][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3200] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.707.916 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.707.936 [graph_prepare.cc:1982][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [61] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.708.665 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [14] [INFO] GE(164040,python):2024-01-10-11:40:27.708.692 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.708.703 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.708.713 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [437] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.708.722 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.708.742 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [14] [INFO] GE(164040,python):2024-01-10-11:40:27.708.751 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [9] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.708.760 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.708.768 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.708.826 [graph_prepare.cc:1983][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [875] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.708.852 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.708.864 [graph_prepare.cc:1984][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.708.878 [graph_prepare.cc:1985][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.708.893 [graph_prepare.cc:1986][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.708.904 [graph_prepare.cc:1987][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.708.919 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.708.931 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.708.946 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.709.069 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.709.081 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.709.090 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.709.099 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.709.107 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.709.115 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.709.124 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.709.132 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.709.141 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.709.149 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.709.166 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.709.175 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.709.184 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.709.192 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [9] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.709.200 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.709.209 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.709.233 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.709.247 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.709.285 [graph_prepare.cc:1988][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [371] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.709.300 [graph_manager.cc:1065][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1427] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.722.231 [graph_manager.cc:1077][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12910] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.722.364 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.722.420 [graph_manager.cc:1080][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [152] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.726.628 [graph_manager.cc:1081][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [4189] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.726.671 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.726.687 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.726.698 [graph_manager.cc:1082][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.726.731 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.726.745 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.726.760 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.726.791 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.726.816 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.726.834 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.726.847 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.726.895 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [39] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.726.915 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.726.945 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.726.981 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.726.998 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.010 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.020 [graph_manager.cc:2700][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [295] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.178 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.727.192 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AddNPass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.727.201 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.727.210 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.727.218 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.727.227 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CastRemovePass is [12] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.727.235 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.727.244 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.727.252 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.727.260 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.727.268 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.727.277 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.727.285 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.727.302 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.727.311 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.727.320 [graph_manager.cc:2741][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [282] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.330 [graph_manager.cc:2752][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.353 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.366 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.386 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.402 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.413 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.426 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.446 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.461 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.473 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.483 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.496 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.508 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.529 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.542 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.551 [graph_manager.cc:2810][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [201] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.589 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.727.602 [graph_manager.cc:2821][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [43] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.629 [graph_manager.cc:1087][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [911] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.767 [graph_manager.cc:1088][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [125] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.823 [graph_manager.cc:1089][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [26] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.842 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.727.885 [graph_manager.cc:1097][EVENT]167129 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:40:27.727.907 [graph_manager.cc:3325][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.728.383 [engine_place.cc:144][EVENT]167129 Run:The time cost of AIcoreEngine::CheckSupported is [365] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.728.411 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.728.421 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.728.514 [graph_manager.cc:3351][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [593] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.728.535 [graph_manager.cc:3364][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.728.604 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [24] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.728.623 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.728.870 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [236] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.728.921 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.728.973 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [41] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.729.010 [graph_manager.cc:3405][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [463] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.729.028 [graph_manager.cc:3412][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.741.561 [graph_manager.cc:3422][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [12517] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.741.602 [graph_manager.cc:3428][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.741.799 [graph_manager.cc:3467][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [178] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.741.820 [graph_manager.cc:3377][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [13274] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.741.846 [graph_manager.cc:1106][EVENT]167129 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [13945] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.741.860 [graph_manager.cc:1115][EVENT]167129 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:40:27.741.883 [graph_manager.cc:1130][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.741.916 [graph_manager.cc:1131][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.741.941 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.741.959 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.741.969 [graph_manager.cc:2837][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.068 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.742.082 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.742.091 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.742.099 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.742.108 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.742.117 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [6] micro second, call num is [7] [INFO] GE(164040,python):2024-01-10-11:40:27.742.126 [graph_manager.cc:2864][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [141] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.139 [graph_manager.cc:2872][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.158 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.173 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.190 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.203 [compile_nodes_pass.cc:88][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.213 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.222 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.343 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [104] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.374 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.387 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.401 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.414 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.424 [graph_manager.cc:2927][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [269] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.436 [graph_manager.cc:2937][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.451 [graph_manager.cc:2943][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.463 [graph_manager.cc:2950][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.643 [graph_manager.cc:2958][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [50] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.677 [graph_manager.cc:1132][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [748] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.748 [graph_manager.cc:1135][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [59] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.787 [graph_manager.cc:2975][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.819 [graph_manager.cc:2981][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.833 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.843 [graph_manager.cc:2986][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.742.852 [graph_manager.cc:1136][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [89] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.743.002 [graph_manager.cc:3555][EVENT]167129 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [113] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.743.111 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.743.128 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.743.320 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [181] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.743.371 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.743.415 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.743.439 [graph_builder.cc:865][EVENT]167129 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [372] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:40:27.743.770 [logger.cc:1071] 167129 ModelBindStream: model_id=1344, stream_id=1601, flag=0. [INFO] GE(164040,python):2024-01-10-11:40:27.743.802 [task_generator.cc:804][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [76] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.743.878 [task_generator.cc:805][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [63] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.744.679 [task_generator.cc:814][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [787] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.744.694 [task_generator.cc:954][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [969] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.744.753 [task_generator.cc:967][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [34] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:40:27.744.772 [logger.cc:1084] 167129 ModelUnbindStream: model_id=1344, stream_id=1601, [INFO] GE(164040,python):2024-01-10-11:40:27.744.952 [graph_manager.cc:1152][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2069] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.744.971 [graph_manager.cc:1164][EVENT]167129 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:40:27.745.002 [graph_manager.cc:1271][EVENT]167129 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [41472] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.745.014 [graph_manager.cc:1272][EVENT]167129 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:40:27.745.336 [atrace_api.c:93](tid:167129) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:40:27.745.351 [atrace_api.c:95](tid:167129) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:40:27.752.747 [graph_converter.cc:838][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1718] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.752.830 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.753.467 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [615] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.753.756 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [261] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.753.781 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [288] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.753.835 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [43] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.753.870 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.753.902 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.754.126 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [201] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.754.234 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [86] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.754.250 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [104] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.754.287 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.754.318 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.754.349 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.754.461 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [102] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.754.552 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [78] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.754.565 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [91] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.754.600 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [26] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.754.629 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.754.644 [graph_converter.cc:849][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1856] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.754.945 [graph_converter.cc:853][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [291] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.755.963 [graph_converter.cc:857][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [998] micro second. [INFO] GE(164040,python):2024-01-10-11:40:27.756.180 [graph_converter.cc:862][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [188] micro second. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:27.760.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 1, total running count: 1 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:27.761.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:27.763.584 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 102 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:27.780.224 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 2, total running count: 2 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:27.780.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:27.783.018 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 103 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:27.795.466 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 3, total running count: 3 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:27.795.833 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:27.797.544 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 104 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:27.810.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 4, total running count: 4 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:27.810.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:27.813.421 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 105 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:27.825.587 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 5, total running count: 5 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:27.826.037 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:27.828.168 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 106 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:27.840.698 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 6, total running count: 6 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:27.841.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:27.843.617 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 107 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:27.855.970 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 7, total running count: 7 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:27.856.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:27.859.476 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 108 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:27.871.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 8, total running count: 8 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:27.872.054 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:27.873.954 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 109 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:27.886.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 9, total running count: 9 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:27.887.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:27.889.801 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 110 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:27.902.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 10, total running count: 10 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:27.902.638 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:27.904.264 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 111 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:27.917.210 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 11, total running count: 11 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:27.917.730 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:27.919.903 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 112 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:27.932.037 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 12, total running count: 12 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:27.932.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:27.934.391 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 113 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:27.946.942 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 13, total running count: 13 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:27.947.779 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:27.949.728 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 114 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:27.961.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 14, total running count: 14 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:27.962.785 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:27.965.266 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 115 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:27.976.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 15, total running count: 15 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:27.977.721 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:27.979.545 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 116 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:27.992.016 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 16, total running count: 16 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:27.992.717 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:27.995.029 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 117 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.006.982 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 17, total running count: 17 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.007.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.010.378 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 118 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.022.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 18, total running count: 18 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.022.707 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.024.712 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 119 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:28.037.015 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 19, total running count: 19 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:28.037.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.040.018 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 120 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.051.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 20, total running count: 20 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.052.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.054.498 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 121 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.066.718 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 21, total running count: 21 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.067.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.069.799 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 122 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.081.937 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 22, total running count: 22 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.082.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.085.302 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 123 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.096.757 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 23, total running count: 23 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.097.407 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.099.622 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 124 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.111.708 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 24, total running count: 24 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.112.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.114.897 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 125 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.126.689 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 25, total running count: 25 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.127.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.129.290 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 126 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.141.421 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 26, total running count: 26 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.142.186 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.144.809 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 127 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:28.156.254 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 27, total running count: 27 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.157.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.158.938 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 128 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.171.278 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 28, total running count: 28 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.171.975 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.174.671 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 129 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.186.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 29, total running count: 29 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.186.883 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.189.001 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 130 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.200.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 30, total running count: 30 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.201.820 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.203.774 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 131 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.216.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 31, total running count: 31 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:28.216.708 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.218.499 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 132 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:28.231.195 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 32, total running count: 32 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.231.619 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.234.201 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 133 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.245.926 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 33, total running count: 33 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.246.734 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.248.491 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 134 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.260.949 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 34, total running count: 34 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.261.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.264.100 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 135 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:28.276.153 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 35, total running count: 35 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.276.587 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.278.239 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 136 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:28.290.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 36, total running count: 36 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.291.532 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.293.770 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 137 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.305.840 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 37, total running count: 37 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:28.306.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.308.092 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 138 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.320.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 38, total running count: 38 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.321.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.323.611 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 139 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.335.776 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 39, total running count: 39 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.336.286 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.338.153 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 140 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.350.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 40, total running count: 40 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.351.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.354.005 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 141 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.365.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 41, total running count: 41 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.366.403 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.368.747 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 142 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.380.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 42, total running count: 42 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.381.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.383.059 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 143 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.395.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 43, total running count: 43 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.396.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.398.458 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 144 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.410.811 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 44, total running count: 44 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.411.423 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.414.005 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 145 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.425.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 45, total running count: 45 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:28.426.471 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.428.479 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 146 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.440.568 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 46, total running count: 46 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.441.358 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.442.974 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 147 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.455.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 47, total running count: 47 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:28.456.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.458.730 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 148 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.470.608 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 48, total running count: 48 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.471.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.473.085 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 149 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:28.485.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 49, total running count: 49 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.486.441 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.488.684 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 150 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.500.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 50, total running count: 50 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.501.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.504.117 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 151 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:28.515.785 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 51, total running count: 51 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.516.583 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.518.871 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 152 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.530.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 52, total running count: 52 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.531.697 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.533.612 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 153 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.545.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 53, total running count: 53 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.546.878 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.549.100 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 154 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:28.561.054 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 54, total running count: 54 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.561.817 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.563.457 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 155 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.576.102 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 55, total running count: 55 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:28.576.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.579.055 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 156 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.590.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 56, total running count: 56 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.591.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.594.343 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 157 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:28.605.993 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 57, total running count: 57 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.606.817 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.608.523 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 158 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.620.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 58, total running count: 58 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:28.621.845 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.623.997 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 159 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.636.042 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 59, total running count: 59 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.636.883 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.639.432 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 160 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.651.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 60, total running count: 60 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.651.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.653.557 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 161 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.665.794 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 61, total running count: 61 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.666.704 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.669.140 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 162 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.680.966 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 62, total running count: 62 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.681.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.683.293 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 163 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.696.073 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 63, total running count: 63 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:28.696.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.698.807 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 164 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.710.840 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 64, total running count: 64 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.711.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.713.098 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 165 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.725.654 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 65, total running count: 65 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.726.653 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.728.566 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 166 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.740.709 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 66, total running count: 66 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.741.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.743.166 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 167 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:28.755.839 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 67, total running count: 67 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:28.756.522 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.759.076 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 168 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.770.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 68, total running count: 68 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.771.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.773.446 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 169 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.785.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 69, total running count: 69 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.786.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.787.936 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 170 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.800.584 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 70, total running count: 70 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.801.176 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.803.232 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 171 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.815.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 71, total running count: 71 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.816.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.818.660 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 172 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.830.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 72, total running count: 72 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.830.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.833.301 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 173 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.844.966 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 73, total running count: 73 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.845.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.847.922 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 174 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.860.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 74, total running count: 74 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.860.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.863.449 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 175 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.875.219 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 75, total running count: 75 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.875.878 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.877.781 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 176 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.890.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 76, total running count: 76 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.891.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.893.093 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 177 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.905.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 77, total running count: 77 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.905.878 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.908.609 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 178 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.920.233 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 78, total running count: 78 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.920.827 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.922.945 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 179 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.935.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 79, total running count: 79 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.935.817 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.938.315 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 180 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.950.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 80, total running count: 80 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.950.732 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.952.577 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 181 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.964.900 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 81, total running count: 81 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:28.965.708 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.968.069 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 182 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:28.979.865 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 82, total running count: 82 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:28.980.734 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.982.335 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 183 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:28.995.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 83, total running count: 83 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:28.995.768 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:28.997.830 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 184 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.010.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 84, total running count: 84 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:29.010.602 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.012.075 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 185 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.024.946 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 85, total running count: 85 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.025.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.027.380 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 186 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.039.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 86, total running count: 86 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:29.040.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.042.840 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 187 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.054.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 87, total running count: 87 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.055.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.057.029 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 188 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.069.856 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 88, total running count: 88 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.070.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.072.583 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 189 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:29.084.395 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 89, total running count: 89 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:29.085.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.088.050 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 190 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.099.709 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 90, total running count: 90 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.100.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.102.756 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 191 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.114.817 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 91, total running count: 91 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:29.115.388 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.117.254 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 192 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:29.129.627 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 92, total running count: 92 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.130.391 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.132.996 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 193 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.144.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 93, total running count: 93 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.145.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.147.513 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 194 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.159.521 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 94, total running count: 94 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.160.425 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.162.099 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 195 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:29.174.625 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 95, total running count: 95 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.175.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.176.921 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 196 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.189.785 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 96, total running count: 96 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.190.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.192.383 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 197 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.204.599 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 97, total running count: 97 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.205.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.206.646 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 198 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.219.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 98, total running count: 98 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.220.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.222.234 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 199 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.234.380 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 99, total running count: 99 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.235.008 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.237.802 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 200 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.249.218 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 100, total running count: 100 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.249.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.252.020 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 201 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.264.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 101, total running count: 101 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.264.813 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.266.324 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 202 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.279.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 102, total running count: 102 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.279.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.281.784 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 203 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.294.162 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 103, total running count: 103 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.294.775 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.297.385 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 204 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.309.102 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 104, total running count: 104 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.309.853 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.311.868 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 205 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.323.887 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 105, total running count: 105 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.324.698 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.326.628 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 206 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:29.339.128 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 106, total running count: 106 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.339.885 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.342.228 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 207 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.354.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 107, total running count: 107 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.354.702 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.356.734 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 208 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.368.925 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 108, total running count: 108 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.369.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.372.231 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 209 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:29.383.899 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 109, total running count: 109 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.384.597 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.386.530 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 210 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.398.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 110, total running count: 110 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.399.462 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.401.042 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 211 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.413.813 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 111, total running count: 111 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.414.516 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.416.581 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 212 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:29.428.679 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 112, total running count: 112 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.429.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.431.136 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 213 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.443.831 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 113, total running count: 113 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.444.521 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.447.072 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 214 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.458.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 114, total running count: 114 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.459.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.461.388 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 215 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.473.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 115, total running count: 115 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.474.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.476.963 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 216 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:29.488.485 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 116, total running count: 116 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.489.455 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.491.385 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 217 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.503.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 117, total running count: 117 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.504.462 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.507.145 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 218 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.518.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 118, total running count: 118 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.519.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.521.785 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 219 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:29.533.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 119, total running count: 119 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.534.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.536.379 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 220 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.548.598 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 120, total running count: 120 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.549.269 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.551.934 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 221 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.563.574 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 121, total running count: 121 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.564.208 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.566.379 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 222 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.578.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 122, total running count: 122 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.579.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.580.989 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 223 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.593.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 123, total running count: 123 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.594.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.595.759 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 224 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.608.398 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 124, total running count: 124 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.608.949 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.611.331 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 225 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.623.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 125, total running count: 125 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.624.002 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.625.570 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 226 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.638.169 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 126, total running count: 126 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.638.997 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.641.155 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 227 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.653.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 127, total running count: 127 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.654.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.656.595 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 228 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.668.349 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 128, total running count: 128 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:29.668.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.670.864 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 229 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.683.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 129, total running count: 129 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.684.200 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.686.169 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 230 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.698.728 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 130, total running count: 130 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.699.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.701.801 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 231 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:29.713.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 131, total running count: 131 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.714.344 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.716.239 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 232 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.728.402 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 132, total running count: 132 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:29.729.356 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.731.585 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 233 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.743.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 133, total running count: 133 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.744.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.746.290 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 234 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.758.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 134, total running count: 134 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.759.281 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.761.005 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 235 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.773.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 135, total running count: 135 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.774.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.776.749 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 236 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:29.788.677 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 136, total running count: 136 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.789.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.791.290 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 237 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.803.853 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 137, total running count: 137 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.804.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.805.883 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 238 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.818.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 138, total running count: 138 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.819.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.821.486 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 239 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.833.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 139, total running count: 139 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.834.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.836.919 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 240 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.848.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 140, total running count: 140 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.849.356 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.851.246 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 241 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.863.295 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 141, total running count: 141 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.864.232 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.866.697 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 242 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.878.442 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 142, total running count: 142 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.884.356 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.885.977 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 243 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:29.898.663 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 143, total running count: 143 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.899.408 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.901.972 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 244 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:29.913.619 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 144, total running count: 144 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.914.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.916.556 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 245 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.928.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 145, total running count: 145 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.929.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.930.924 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 246 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:29.943.535 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 146, total running count: 146 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.944.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.946.418 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 247 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.958.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 147, total running count: 147 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:29.959.097 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.960.602 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 248 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:29.973.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 148, total running count: 148 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.974.018 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.976.012 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 249 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:29.988.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 149, total running count: 149 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:29.989.013 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:29.991.366 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 250 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.003.374 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 150, total running count: 150 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.003.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.005.746 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 251 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.018.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 151, total running count: 151 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.019.073 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.021.339 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 252 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.033.348 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 152, total running count: 152 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.034.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.036.745 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 253 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.048.290 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 153, total running count: 153 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.049.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.050.948 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 254 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.063.752 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 154, total running count: 154 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.064.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.066.376 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 255 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.078.361 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 155, total running count: 155 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.079.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.081.670 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 256 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.093.387 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 156, total running count: 156 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.094.124 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.095.919 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 257 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.108.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 157, total running count: 157 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.109.019 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.111.430 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 258 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.123.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 158, total running count: 158 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.124.043 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.125.792 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 259 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.138.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 159, total running count: 159 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.139.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.141.341 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 260 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.153.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 160, total running count: 160 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.154.092 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.156.692 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 261 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.168.620 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 161, total running count: 161 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.169.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.170.940 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 262 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.183.343 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 162, total running count: 162 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.184.207 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.186.491 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 263 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.198.576 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 163, total running count: 163 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.199.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.201.859 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 264 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.213.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 164, total running count: 164 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.214.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.216.165 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 265 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.228.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 165, total running count: 165 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.229.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.231.537 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 266 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.243.663 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 166, total running count: 166 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.244.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.246.141 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 267 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.258.388 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 167, total running count: 167 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.259.345 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.261.883 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 268 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.273.558 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 168, total running count: 168 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.274.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.276.620 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 269 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.288.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 169, total running count: 169 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.289.300 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.290.926 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 270 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.303.560 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 170, total running count: 170 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.304.281 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.306.421 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 271 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.318.563 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 171, total running count: 171 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.319.427 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.321.753 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 272 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.333.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 172, total running count: 172 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.334.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.336.044 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 273 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.348.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 173, total running count: 173 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.349.362 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.351.411 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 274 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.363.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 174, total running count: 174 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.364.395 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.366.903 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 275 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.378.670 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 175, total running count: 175 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.379.213 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.381.342 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 276 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.393.510 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 176, total running count: 176 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.394.214 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.396.871 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 277 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.408.701 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 177, total running count: 177 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.409.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.411.223 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 278 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.423.696 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 178, total running count: 178 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.424.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.426.785 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 279 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.438.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 179, total running count: 179 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.439.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.441.259 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 280 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.453.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 180, total running count: 180 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.454.287 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.455.939 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 281 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.468.575 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 181, total running count: 181 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.469.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.471.393 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 282 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.483.408 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 182, total running count: 182 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.484.182 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.486.834 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 283 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.498.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 183, total running count: 183 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.499.140 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.501.272 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 284 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.513.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 184, total running count: 184 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.514.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.516.490 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 285 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.528.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 185, total running count: 185 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.529.131 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.530.880 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 286 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.543.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 186, total running count: 186 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.543.944 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.546.487 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 287 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.558.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 187, total running count: 187 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.558.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.560.685 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 288 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.573.265 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 188, total running count: 188 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.573.809 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.576.082 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 289 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.587.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 189, total running count: 189 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.588.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.590.343 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 290 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.603.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 190, total running count: 190 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.603.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.605.630 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 291 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.617.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 191, total running count: 191 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.618.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.621.098 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 292 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.632.720 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 192, total running count: 192 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.633.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.635.530 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 293 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.647.479 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 193, total running count: 193 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.648.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.649.827 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 294 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.662.432 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 194, total running count: 194 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.663.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.665.499 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 295 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.677.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 195, total running count: 195 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.678.181 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.680.121 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 296 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.692.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 196, total running count: 196 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.693.154 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.695.720 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 297 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.707.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 197, total running count: 197 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.708.032 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.710.084 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 298 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.722.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 198, total running count: 198 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.723.077 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.725.749 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 299 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.737.419 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 199, total running count: 199 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.737.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.740.010 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 300 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.752.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 200, total running count: 200 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.752.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.755.343 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 301 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.767.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 201, total running count: 201 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.767.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.769.825 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 302 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.782.030 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 202, total running count: 202 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.782.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.785.511 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 303 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.797.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 203, total running count: 203 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.797.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.800.052 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 304 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.811.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 204, total running count: 204 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.812.624 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.814.571 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 305 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.827.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 205, total running count: 205 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.827.468 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.830.035 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 306 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.841.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 206, total running count: 206 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.842.498 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.844.505 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 307 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:30.856.690 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 207, total running count: 207 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.857.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.859.834 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 308 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.871.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 208, total running count: 208 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.872.068 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.874.153 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 309 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.886.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 209, total running count: 209 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.887.065 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.889.662 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 310 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.901.303 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 210, total running count: 210 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.902.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.904.060 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 311 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.916.325 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 211, total running count: 211 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.917.009 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.918.596 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 312 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.931.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 212, total running count: 212 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.931.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.934.425 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 313 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.946.122 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 213, total running count: 213 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:30.946.792 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.948.648 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 314 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.961.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 214, total running count: 214 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.961.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.963.889 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 315 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:30.976.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 215, total running count: 215 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.976.625 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.978.229 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 316 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:30.990.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 216, total running count: 216 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:30.991.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:30.993.693 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 317 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.005.879 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 217, total running count: 217 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.006.560 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.009.077 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 318 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.020.988 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 218, total running count: 218 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.021.732 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.023.301 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 319 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.036.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 219, total running count: 219 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.036.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.038.699 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 320 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.051.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 220, total running count: 220 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.051.444 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.053.185 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 321 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.065.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 221, total running count: 221 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.066.485 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.068.159 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 322 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.080.713 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 222, total running count: 222 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.081.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.083.809 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 323 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.095.598 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 223, total running count: 223 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.096.270 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.098.386 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 324 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.110.589 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 224, total running count: 224 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.111.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.112.892 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 325 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.125.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 225, total running count: 225 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.126.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.128.755 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 326 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:31.140.458 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 226, total running count: 226 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.141.171 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.143.241 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 327 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.155.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 227, total running count: 227 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.156.149 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.157.775 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 328 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.170.326 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 228, total running count: 228 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.171.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.173.788 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 329 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:31.185.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 229, total running count: 229 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.186.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.188.306 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 330 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.200.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 230, total running count: 230 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.201.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.203.772 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 331 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.215.408 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 231, total running count: 231 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.216.105 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.218.412 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 332 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:31.230.462 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 232, total running count: 232 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.231.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.232.906 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 333 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:31.245.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 233, total running count: 233 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.245.955 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.248.486 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 334 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.260.224 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 234, total running count: 234 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.261.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.262.831 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 335 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.275.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 235, total running count: 235 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.275.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.278.360 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 336 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.290.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 236, total running count: 236 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.291.068 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.293.787 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 337 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.305.330 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 237, total running count: 237 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.306.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.308.011 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 338 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.320.575 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 238, total running count: 238 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:31.321.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.323.855 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 339 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.335.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 239, total running count: 239 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.336.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.338.542 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 340 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.350.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 240, total running count: 240 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.351.347 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.353.023 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 341 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.365.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 241, total running count: 241 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.366.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.368.520 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 342 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.380.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 242, total running count: 242 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.381.496 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.384.031 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 343 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.395.853 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 243, total running count: 243 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.396.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.398.306 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 344 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.410.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 244, total running count: 244 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:31.411.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.413.811 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 345 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:31.425.737 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 245, total running count: 245 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:31.426.495 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.428.176 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 346 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.440.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 246, total running count: 246 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.441.432 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.443.944 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 347 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.455.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 247, total running count: 247 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.456.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.458.657 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 348 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.470.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 248, total running count: 248 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:31.471.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.473.023 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 349 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:31.485.609 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 249, total running count: 249 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.486.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.488.715 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 350 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.500.653 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 250, total running count: 250 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.501.246 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.503.437 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 351 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.515.689 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 251, total running count: 251 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.516.303 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.517.880 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 352 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:31.530.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 252, total running count: 252 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.531.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.533.326 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 353 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.545.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 253, total running count: 253 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.546.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.548.668 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 354 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.560.681 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 254, total running count: 254 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.561.419 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.564.115 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 355 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.575.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 255, total running count: 255 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.576.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.578.548 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 356 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.591.002 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 256, total running count: 256 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.591.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.593.916 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 357 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.605.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 257, total running count: 257 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:31.607.066 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.609.317 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 358 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.621.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 258, total running count: 258 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.622.132 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.624.747 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 359 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.636.806 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 259, total running count: 259 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.637.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.639.050 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 360 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:31.651.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 260, total running count: 260 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.652.412 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.654.675 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 361 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.666.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 261, total running count: 261 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.667.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.670.201 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 362 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.681.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 262, total running count: 262 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.682.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.684.645 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 363 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:31.696.809 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 263, total running count: 263 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.697.618 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.700.267 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 364 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.711.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 264, total running count: 264 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.712.670 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.714.951 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 365 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:31.727.027 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 265, total running count: 265 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:31.727.694 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.729.563 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 366 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.742.111 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 266, total running count: 266 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.742.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.745.051 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 367 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.756.989 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 267, total running count: 267 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.757.644 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.759.424 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 368 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.772.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 268, total running count: 268 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.772.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.775.296 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 369 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.787.124 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 269, total running count: 269 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.787.589 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.789.810 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 370 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.802.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 270, total running count: 270 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.802.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.804.339 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 371 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.817.032 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 271, total running count: 271 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.817.554 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.820.179 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 372 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.831.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 272, total running count: 272 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.832.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.834.627 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 373 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:31.846.822 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 273, total running count: 273 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:31.847.525 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.850.045 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 374 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:31.861.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 274, total running count: 274 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.862.562 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.864.513 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 375 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.876.942 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 275, total running count: 275 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.877.523 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.880.120 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 376 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.891.708 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 276, total running count: 276 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.892.471 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.894.954 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 377 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.906.485 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 277, total running count: 277 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.907.574 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.909.483 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 378 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.921.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 278, total running count: 278 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.922.700 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.924.909 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 379 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.936.755 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 279, total running count: 279 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.937.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.940.492 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 380 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.952.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 280, total running count: 280 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.952.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.954.765 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 381 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:31.967.044 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 281, total running count: 281 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.967.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.970.238 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 382 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:31.982.127 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 282, total running count: 282 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.982.683 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:31.984.477 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 383 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:31.997.048 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 283, total running count: 283 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:31.997.654 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.000.246 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 384 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.011.877 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 284, total running count: 284 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.012.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.014.865 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 385 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.027.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 285, total running count: 285 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.027.554 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.029.385 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 386 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.041.780 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 286, total running count: 286 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.042.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.044.076 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 387 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.056.978 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 287, total running count: 287 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.057.417 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.059.815 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 388 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.071.736 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 288, total running count: 288 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.072.300 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.074.553 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 389 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.086.582 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 289, total running count: 289 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.087.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.089.298 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 390 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.101.647 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 290, total running count: 290 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.102.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.103.861 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 391 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.116.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 291, total running count: 291 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.117.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.119.737 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 392 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.131.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 292, total running count: 292 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.131.955 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.134.220 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 393 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.146.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 293, total running count: 293 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.146.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.148.641 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 394 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.161.053 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 294, total running count: 294 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.161.733 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.164.126 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 395 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.176.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 295, total running count: 295 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.176.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.178.823 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 396 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.191.072 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 296, total running count: 296 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.191.594 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.193.492 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 397 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.205.812 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 297, total running count: 297 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.206.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.208.976 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 398 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.220.968 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 298, total running count: 298 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.221.594 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.223.305 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 399 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.236.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 299, total running count: 299 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.236.653 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.238.720 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 400 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.250.794 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 300, total running count: 300 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.251.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.254.152 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 401 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.265.804 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 301, total running count: 301 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.266.640 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.268.547 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 402 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.281.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 302, total running count: 302 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.281.619 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.283.853 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 403 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.296.035 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 303, total running count: 303 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.296.759 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.299.311 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 404 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.310.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 304, total running count: 304 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.311.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.314.015 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 405 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.325.944 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 305, total running count: 305 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.326.674 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.329.453 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 406 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.340.968 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 306, total running count: 306 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.341.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.343.839 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 407 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.355.805 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 307, total running count: 307 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.356.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.358.407 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 408 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.370.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 308, total running count: 308 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.371.398 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.373.935 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 409 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.385.733 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 309, total running count: 309 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.386.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.388.587 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 410 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.400.689 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 310, total running count: 310 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.401.447 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.403.121 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 411 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.415.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 311, total running count: 311 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.416.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.418.859 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 412 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.430.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 312, total running count: 312 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.431.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.433.334 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 413 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.445.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 313, total running count: 313 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.446.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.447.987 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 414 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.460.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 314, total running count: 314 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.461.314 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.463.833 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 415 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.475.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 315, total running count: 315 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.476.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.478.256 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 416 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.490.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 316, total running count: 316 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.491.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.493.609 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 417 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.505.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 317, total running count: 317 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.506.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.508.247 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 418 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.520.717 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 318, total running count: 318 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.521.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.523.882 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 419 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.535.275 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 319, total running count: 319 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.536.210 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.538.612 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 420 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.550.523 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 320, total running count: 320 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.551.028 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.552.957 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 421 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.565.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 321, total running count: 321 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.566.146 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.568.601 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 422 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.580.415 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 322, total running count: 322 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.581.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.583.090 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 423 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.595.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 323, total running count: 323 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.596.134 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.597.721 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 424 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.610.352 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 324, total running count: 324 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.611.140 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.613.362 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 425 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.625.370 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 325, total running count: 325 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.625.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.627.737 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 426 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.640.608 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 326, total running count: 326 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.641.217 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.643.866 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 427 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.655.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 327, total running count: 327 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.656.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.658.657 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 428 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.671.072 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 328, total running count: 328 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.671.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.674.502 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 429 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.686.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 329, total running count: 329 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.687.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.689.318 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 430 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.701.822 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 330, total running count: 330 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.702.522 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.704.250 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 431 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.717.096 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 331, total running count: 331 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.717.681 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.720.417 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 432 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.732.030 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 332, total running count: 332 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.732.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.735.224 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 433 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.747.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 333, total running count: 333 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.747.872 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.750.025 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 434 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.762.614 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 334, total running count: 334 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.763.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.766.135 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 435 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.777.731 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 335, total running count: 335 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.778.300 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.780.952 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 436 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.792.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 336, total running count: 336 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.793.238 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.795.568 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 437 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.807.916 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 337, total running count: 337 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.808.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.810.257 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 438 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.823.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 338, total running count: 338 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.823.766 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.826.313 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 439 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.838.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 339, total running count: 339 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.838.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.841.323 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 440 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.853.404 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 340, total running count: 340 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.854.019 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.856.276 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 441 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.868.640 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 341, total running count: 341 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.869.053 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.871.344 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 442 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.883.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 342, total running count: 342 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.884.262 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.886.477 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 443 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.898.851 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 343, total running count: 343 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.899.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.901.923 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 444 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.913.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 344, total running count: 344 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.914.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.917.007 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 445 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.928.808 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 345, total running count: 345 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.929.437 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.931.401 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 446 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.944.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 346, total running count: 346 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:32.944.413 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.946.447 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 447 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:32.958.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 347, total running count: 347 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.959.698 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.962.533 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 448 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.974.260 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 348, total running count: 348 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:32.974.702 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.976.717 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 449 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:32.989.413 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 349, total running count: 349 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:32.990.013 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:32.991.650 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 450 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.004.374 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 350, total running count: 350 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.004.851 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.007.395 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 451 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.019.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 351, total running count: 351 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.019.790 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.022.423 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 452 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.034.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 352, total running count: 352 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.034.775 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.037.064 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 453 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.049.344 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 353, total running count: 353 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.049.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.051.536 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 454 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.064.168 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 354, total running count: 354 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.064.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.067.215 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 455 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.079.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 355, total running count: 355 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.079.858 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.081.761 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 456 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.094.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 356, total running count: 356 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.094.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.097.260 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 457 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.109.013 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 357, total running count: 357 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.109.939 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.112.073 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 458 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.124.308 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 358, total running count: 358 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.125.086 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.127.582 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 459 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.139.527 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 359, total running count: 359 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.140.052 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.141.756 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 460 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.154.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 360, total running count: 360 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.154.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.156.387 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 461 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.169.171 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 361, total running count: 361 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.170.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.172.022 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 462 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.184.465 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 362, total running count: 362 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.185.116 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.187.610 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 463 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.199.638 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 363, total running count: 363 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.200.096 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.202.121 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 464 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.214.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 364, total running count: 364 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.215.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.216.979 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 465 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.229.437 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 365, total running count: 365 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.229.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.232.831 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 466 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.244.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 366, total running count: 366 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.245.207 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.247.447 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 467 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.259.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 367, total running count: 367 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.260.287 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.262.074 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 468 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.274.617 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 368, total running count: 368 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.275.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.277.887 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 469 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.289.508 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 369, total running count: 369 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.290.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.292.694 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 470 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.304.759 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 370, total running count: 370 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.305.224 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.307.079 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 471 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.319.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 371, total running count: 371 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.320.250 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.321.977 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 472 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.334.717 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 372, total running count: 372 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.335.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.337.567 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 473 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.349.993 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 373, total running count: 373 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.350.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.353.129 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 474 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.364.707 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 374, total running count: 374 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.365.308 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.367.514 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 475 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.379.489 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 375, total running count: 375 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.380.192 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.381.850 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 476 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.394.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 376, total running count: 376 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.395.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.397.473 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 477 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.409.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 377, total running count: 377 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.410.262 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.411.876 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 478 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.424.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 378, total running count: 378 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.425.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.427.620 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 479 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.439.722 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 379, total running count: 379 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.440.114 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.442.982 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 480 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.454.478 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 380, total running count: 380 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.455.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.457.060 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 481 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.469.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 381, total running count: 381 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.470.068 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.472.659 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 482 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.484.399 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 382, total running count: 382 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.485.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.487.014 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 483 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.499.871 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 383, total running count: 383 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.500.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.502.694 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 484 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.514.508 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 384, total running count: 384 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.514.840 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.517.168 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 485 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.529.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 385, total running count: 385 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.529.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.531.580 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 486 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.544.216 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 386, total running count: 386 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.544.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.546.209 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 487 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.558.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 387, total running count: 387 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.559.375 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.560.966 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 488 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.573.657 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 388, total running count: 388 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.574.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.576.709 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 489 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.588.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 389, total running count: 389 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.589.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.591.564 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 490 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.603.780 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 390, total running count: 390 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.604.482 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.606.326 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 491 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.618.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 391, total running count: 391 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.619.501 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.621.894 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 492 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.634.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 392, total running count: 392 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.634.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.636.537 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 493 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.649.024 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 393, total running count: 393 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.649.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.651.933 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 494 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.663.663 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 394, total running count: 394 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.664.474 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.666.221 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 495 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.679.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 395, total running count: 395 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.679.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.681.790 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 496 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.693.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 396, total running count: 396 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.694.473 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.696.117 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 497 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.708.975 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 397, total running count: 397 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.709.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.711.208 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 498 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.723.693 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 398, total running count: 398 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.724.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.727.014 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 499 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.738.661 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 399, total running count: 399 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.739.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.741.618 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 500 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.753.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 400, total running count: 400 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.754.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.756.354 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 501 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.768.559 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 401, total running count: 401 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.769.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.771.183 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 502 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.783.459 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 402, total running count: 402 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.783.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.786.444 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 503 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.798.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 403, total running count: 403 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.799.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.801.415 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 504 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.813.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 404, total running count: 404 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.813.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.815.775 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 505 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.828.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 405, total running count: 405 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.828.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.830.332 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 506 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.842.982 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 406, total running count: 406 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.843.820 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.846.221 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 507 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.858.015 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 407, total running count: 407 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.858.594 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.860.755 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 508 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:33.872.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 408, total running count: 408 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.873.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.876.325 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 509 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.887.937 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 409, total running count: 409 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.888.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.891.001 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 510 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.902.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 410, total running count: 410 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.903.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.905.609 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 511 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.917.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 411, total running count: 411 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.918.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.920.201 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 512 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.932.597 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 412, total running count: 412 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.933.102 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.935.810 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 513 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.947.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 413, total running count: 413 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.947.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.949.789 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 514 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:33.962.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 414, total running count: 414 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.962.854 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.965.473 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 515 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.977.091 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 415, total running count: 415 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:33.977.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.980.229 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 516 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:33.992.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 416, total running count: 416 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:33.992.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:33.994.741 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 517 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.007.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 417, total running count: 417 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.007.877 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.009.583 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 518 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.022.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 418, total running count: 418 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.022.812 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.025.184 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 519 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.037.162 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 419, total running count: 419 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.037.629 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.039.817 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 520 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.051.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 420, total running count: 420 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.052.613 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.054.216 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 521 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.066.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 421, total running count: 421 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.067.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.069.787 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 522 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.082.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 422, total running count: 422 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.082.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.084.653 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 523 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.096.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 423, total running count: 423 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.097.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.099.107 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 524 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.111.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 424, total running count: 424 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.112.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.114.863 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 525 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.126.661 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 425, total running count: 425 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.127.238 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.129.261 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 526 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.141.515 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 426, total running count: 426 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.142.247 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.144.777 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 527 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.156.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 427, total running count: 427 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.157.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.159.342 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 528 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.171.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 428, total running count: 428 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.171.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.174.271 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 529 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.186.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 429, total running count: 429 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.186.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.189.031 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 530 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.200.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 430, total running count: 430 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.201.805 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.203.483 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 531 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.215.970 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 431, total running count: 431 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.216.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.219.051 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 532 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.231.142 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 432, total running count: 432 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.231.651 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.233.386 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 533 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.245.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 433, total running count: 433 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.246.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.248.716 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 534 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.261.102 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 434, total running count: 434 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.261.586 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.264.226 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 535 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.275.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 435, total running count: 435 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.276.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.278.396 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 536 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.291.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 436, total running count: 436 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.291.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.293.954 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 537 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.306.078 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 437, total running count: 437 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.306.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.308.572 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 538 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.320.783 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 438, total running count: 438 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.321.281 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.323.353 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 539 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.335.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 439, total running count: 439 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.336.212 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.337.796 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 540 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.350.443 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 440, total running count: 440 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.351.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.353.785 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 541 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.365.788 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 441, total running count: 441 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.366.236 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.368.131 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 542 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.380.582 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 442, total running count: 442 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.381.128 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.383.638 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 543 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.395.576 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 443, total running count: 443 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.395.988 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.398.099 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 544 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.410.375 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 444, total running count: 444 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.410.903 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.412.672 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 545 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.424.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 445, total running count: 445 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.425.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.427.463 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 546 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.440.353 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 446, total running count: 446 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.440.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.443.483 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 547 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.455.112 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 447, total running count: 447 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.455.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.457.847 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 548 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.470.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 448, total running count: 448 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.470.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.473.358 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 549 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.485.044 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 449, total running count: 449 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.485.503 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.488.266 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 550 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.500.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 450, total running count: 450 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.500.535 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.502.836 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 551 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.514.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 451, total running count: 451 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.515.559 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.517.280 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 552 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.529.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 452, total running count: 452 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.530.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.532.880 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 553 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.545.153 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 453, total running count: 453 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.545.551 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.547.306 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 554 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.559.900 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 454, total running count: 454 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.560.535 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.563.116 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 555 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.574.968 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 455, total running count: 455 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.575.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.577.499 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 556 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.590.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 456, total running count: 456 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.590.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.593.212 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 557 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.604.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 457, total running count: 457 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.605.590 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.607.770 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 558 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.619.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 458, total running count: 458 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.620.612 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.622.370 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 559 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.635.115 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 459, total running count: 459 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.635.714 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.638.287 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 560 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.649.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 460, total running count: 460 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.650.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.652.913 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 561 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.664.703 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 461, total running count: 461 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.665.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.667.315 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 562 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.679.749 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 462, total running count: 462 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.680.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.682.906 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 563 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.694.588 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 463, total running count: 463 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.695.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.697.555 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 564 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.709.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 464, total running count: 464 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.709.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.712.098 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 565 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.723.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 465, total running count: 465 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.724.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.726.616 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 566 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.739.080 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 466, total running count: 466 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.739.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.741.370 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 567 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.753.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 467, total running count: 467 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.754.344 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.755.955 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 568 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.768.713 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 468, total running count: 468 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:40:34.769.382 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_1 execution count: 1, execution time: 182586 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:40:34.769.551 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_1 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:40:34.769.709 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty epoch: 1 step: 468, loss is 2.2887206077575684 [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:40:34.770.829 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.h:76] ContinueSend] continue send at the beginning of the epoch [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:40:34.772.149 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:40:34.772.217 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:40:34.772.259 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_1 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.772.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.775.219 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 569 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.787.146 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 1, total running count: 469 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.787.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.789.556 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 570 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.802.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 2, total running count: 470 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.803.019 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.805.272 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 571 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.817.146 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 3, total running count: 471 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.817.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.819.799 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 572 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.832.158 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 4, total running count: 472 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.832.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.835.400 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 573 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.847.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 5, total running count: 473 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.848.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.849.938 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 574 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.862.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 6, total running count: 474 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.863.008 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.864.649 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 575 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.877.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 7, total running count: 475 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.877.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.880.304 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 576 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.892.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 8, total running count: 476 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.893.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.894.780 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 577 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.907.462 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 9, total running count: 477 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.908.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.910.451 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 578 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.922.058 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 10, total running count: 478 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.922.898 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.925.036 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 579 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.937.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 11, total running count: 479 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.937.757 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.939.545 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 580 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:34.952.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 12, total running count: 480 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.952.725 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.954.909 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 581 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.967.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 13, total running count: 481 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.967.724 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.969.920 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 582 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:34.982.058 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 14, total running count: 482 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:34.982.735 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.984.358 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 583 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:34.996.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 15, total running count: 483 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:34.997.570 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:34.999.922 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 584 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.011.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 16, total running count: 484 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.012.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.014.198 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 585 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.026.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 17, total running count: 485 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.027.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.029.668 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 586 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.041.898 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 18, total running count: 486 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.042.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.045.057 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 587 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:35.057.039 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 19, total running count: 487 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.057.501 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.059.404 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 588 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.071.703 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 20, total running count: 488 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.072.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.074.951 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 589 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.086.623 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 21, total running count: 489 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.087.417 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.089.252 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 590 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.101.754 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 22, total running count: 490 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.102.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.104.625 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 591 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.116.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 23, total running count: 491 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.117.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.118.818 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 592 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.131.352 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 24, total running count: 492 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.131.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.134.419 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 593 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.146.372 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 25, total running count: 493 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:35.146.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.148.897 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 594 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:35.161.068 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 26, total running count: 494 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.163.236 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.165.466 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 595 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.177.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 27, total running count: 495 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:35.178.900 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.181.098 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 596 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.193.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 28, total running count: 496 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.193.922 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.195.565 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 597 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.208.140 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 29, total running count: 497 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.208.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.210.829 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 598 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.223.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 30, total running count: 498 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.223.815 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.226.605 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 599 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.238.146 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 31, total running count: 499 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.238.636 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.241.110 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 600 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.253.045 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 32, total running count: 500 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.253.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.255.432 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 601 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.267.789 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 33, total running count: 501 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.268.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.270.883 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 602 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.282.865 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 34, total running count: 502 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.283.510 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.285.342 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 603 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.297.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 35, total running count: 503 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.298.587 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.300.885 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 604 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.312.880 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 36, total running count: 504 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:35.313.580 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.315.473 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 605 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.327.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 37, total running count: 505 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:35.328.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.330.192 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 606 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.342.742 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 38, total running count: 506 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.343.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.345.935 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 607 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.357.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 39, total running count: 507 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.358.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.360.497 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 608 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.372.599 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 40, total running count: 508 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.373.109 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.374.882 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 609 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.387.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 41, total running count: 509 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.387.789 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.389.432 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 610 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.402.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 42, total running count: 510 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.402.589 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.405.315 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 611 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:35.417.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 43, total running count: 511 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.417.764 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.419.633 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 612 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.432.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 44, total running count: 512 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.432.806 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.435.362 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 613 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.447.263 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 45, total running count: 513 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.448.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.450.076 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 614 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.462.412 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 46, total running count: 514 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.463.750 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.465.835 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 615 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:35.478.092 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 47, total running count: 515 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:35.478.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.481.300 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 616 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.493.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 48, total running count: 516 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.493.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.495.755 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 617 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.508.044 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 49, total running count: 517 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.508.640 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.511.287 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 618 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.523.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 50, total running count: 518 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:35.523.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.525.760 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 619 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.538.235 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 51, total running count: 519 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.539.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.542.361 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 620 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.554.135 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 52, total running count: 520 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:35.555.132 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.556.946 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 621 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.569.462 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 53, total running count: 521 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.570.348 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.572.832 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 622 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.584.759 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 54, total running count: 522 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.585.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.587.277 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 623 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.599.864 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 55, total running count: 523 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.600.584 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.602.652 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 624 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.614.948 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 56, total running count: 524 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.615.762 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.618.362 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 625 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.630.037 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 57, total running count: 525 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.630.781 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.633.030 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 626 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.645.046 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 58, total running count: 526 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.645.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.647.557 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 627 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.659.975 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 59, total running count: 527 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.660.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.663.284 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 628 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.675.335 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 60, total running count: 528 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.675.813 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.677.585 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 629 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.690.140 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 61, total running count: 529 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.690.765 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.693.047 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 630 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.704.829 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 62, total running count: 530 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.705.627 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.707.228 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 631 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.719.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 63, total running count: 531 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.720.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.723.016 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 632 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.734.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 64, total running count: 532 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.735.775 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.737.552 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 633 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.750.289 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 65, total running count: 533 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.750.800 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.753.101 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 634 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.765.054 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 66, total running count: 534 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.765.872 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.767.445 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 635 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.780.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 67, total running count: 535 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.784.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.786.206 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 636 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.798.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 68, total running count: 536 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.801.077 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.802.862 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 637 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.815.488 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 69, total running count: 537 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.816.516 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.818.267 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 638 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.830.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 70, total running count: 538 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:35.833.275 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.834.992 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 639 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.847.613 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 71, total running count: 539 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.849.183 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.851.833 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 640 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.863.780 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 72, total running count: 540 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.864.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.866.528 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 641 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.879.013 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 73, total running count: 541 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.879.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.882.288 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 642 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:35.893.969 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 74, total running count: 542 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.894.897 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.896.742 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 643 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.909.178 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 75, total running count: 543 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.910.102 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.912.002 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 644 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.924.379 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 76, total running count: 544 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.925.291 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.927.658 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 645 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.939.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 77, total running count: 545 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:35.940.372 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.942.109 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 646 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.954.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 78, total running count: 546 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.959.612 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.962.267 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 647 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:35.973.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 79, total running count: 547 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:35.974.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.976.581 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 648 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:35.989.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 80, total running count: 548 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:35.990.121 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:35.992.259 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 649 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.004.361 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 81, total running count: 549 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.005.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.006.762 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 650 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.019.381 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 82, total running count: 550 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.020.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.022.321 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 651 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.034.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 83, total running count: 551 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.035.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.037.853 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 652 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.049.567 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 84, total running count: 552 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.050.281 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.052.124 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 653 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.064.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 85, total running count: 553 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:36.065.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.067.519 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 654 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.079.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 86, total running count: 554 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.080.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.082.171 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 655 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.094.670 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 87, total running count: 555 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.097.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.099.793 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 656 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.111.609 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 88, total running count: 556 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.112.474 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.114.297 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 657 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.126.708 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 89, total running count: 557 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.127.618 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.129.839 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 658 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.141.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 90, total running count: 558 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.142.765 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.144.427 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 659 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.157.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 91, total running count: 559 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.157.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.160.120 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 660 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.172.292 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 92, total running count: 560 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.172.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.174.597 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 661 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:36.187.243 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 93, total running count: 561 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.187.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.190.382 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 662 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.202.130 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 94, total running count: 562 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.202.809 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.205.045 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 663 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:36.217.329 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 95, total running count: 563 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.218.043 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.220.676 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 664 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.232.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 96, total running count: 564 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.233.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.234.989 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 665 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.247.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 97, total running count: 565 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.248.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.250.787 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 666 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:36.262.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 98, total running count: 566 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.263.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.265.462 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 667 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.277.793 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 99, total running count: 567 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:36.278.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.280.058 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 668 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.293.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 100, total running count: 568 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.293.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.295.546 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 669 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:36.307.993 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 101, total running count: 569 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.308.354 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.310.946 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 670 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.322.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 102, total running count: 570 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.323.416 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.325.432 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 671 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.337.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 103, total running count: 571 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.338.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.340.985 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 672 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.352.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 104, total running count: 572 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:36.353.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.355.438 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 673 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.367.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 105, total running count: 573 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.368.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.370.172 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 674 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.382.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 106, total running count: 574 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.383.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.385.787 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 675 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.397.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 107, total running count: 575 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.398.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.400.049 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 676 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:36.412.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 108, total running count: 576 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.415.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.416.747 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 677 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.429.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 109, total running count: 577 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.431.007 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.433.517 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 678 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:36.445.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 110, total running count: 578 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.446.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.449.314 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 679 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.461.053 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 111, total running count: 579 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.462.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.464.955 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 680 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.476.989 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 112, total running count: 580 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.478.246 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.480.527 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 681 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.492.596 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 113, total running count: 581 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.493.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.495.846 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 682 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.508.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 114, total running count: 582 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.514.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.516.060 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 683 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.528.278 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 115, total running count: 583 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.531.168 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.532.796 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 684 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.545.488 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 116, total running count: 584 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.546.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.549.358 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 685 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.561.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 117, total running count: 585 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.562.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.565.249 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 686 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.576.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 118, total running count: 586 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.578.335 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.579.967 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 687 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.592.740 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 119, total running count: 587 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.593.816 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.595.509 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 688 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.608.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 120, total running count: 588 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.609.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.612.006 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 689 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.623.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 121, total running count: 589 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.625.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.627.392 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 690 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.639.964 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 122, total running count: 590 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.641.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.642.915 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 691 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.655.207 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 123, total running count: 591 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.657.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.659.546 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 692 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:36.671.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 124, total running count: 592 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.672.725 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.674.942 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 693 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:36.687.207 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 125, total running count: 593 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.688.218 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.690.485 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 694 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:36.702.443 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 126, total running count: 594 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.703.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.706.024 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 695 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.718.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 127, total running count: 595 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.719.614 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.721.555 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 696 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.733.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 128, total running count: 596 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.736.563 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.738.231 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 697 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.750.985 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 129, total running count: 597 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.752.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.753.858 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 698 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.766.545 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 130, total running count: 598 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.767.841 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.769.846 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 699 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:36.782.100 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 131, total running count: 599 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.783.511 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.785.406 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 700 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:36.797.719 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 132, total running count: 600 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.799.254 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.800.980 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 701 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.813.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 133, total running count: 601 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.815.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.817.545 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 702 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.829.362 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 134, total running count: 602 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:36.830.516 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.833.281 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 703 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.845.000 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 135, total running count: 603 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.848.899 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.850.854 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 704 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.863.317 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 136, total running count: 604 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.864.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.867.299 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 705 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.879.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 137, total running count: 605 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.881.122 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.882.743 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 706 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.895.556 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 138, total running count: 606 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.896.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.899.335 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 707 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:36.911.335 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 139, total running count: 607 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.912.558 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.914.729 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 708 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.926.827 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 140, total running count: 608 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.928.425 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.930.205 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 709 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.942.943 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 141, total running count: 609 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.949.171 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.951.385 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 710 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:36.963.465 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 142, total running count: 610 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:36.964.632 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.966.727 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 711 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.978.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 143, total running count: 611 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:36.980.324 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.982.278 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 712 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:36.994.612 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 144, total running count: 612 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:36.996.079 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:36.997.882 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 713 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.010.473 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 145, total running count: 613 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.011.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.013.457 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 714 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.025.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 146, total running count: 614 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.027.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.029.288 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 715 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.042.091 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 147, total running count: 615 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.043.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.044.930 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 716 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.057.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 148, total running count: 616 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.058.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.060.464 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 717 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.073.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 149, total running count: 617 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.074.697 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.077.307 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 718 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.089.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 150, total running count: 618 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.090.592 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.093.071 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 719 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.105.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 151, total running count: 619 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.106.677 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.108.911 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 720 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.121.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 152, total running count: 620 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.122.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.124.463 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 721 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.136.677 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 153, total running count: 621 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.138.151 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.139.723 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 722 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.152.543 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 154, total running count: 622 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.153.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.156.303 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 723 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.168.217 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 155, total running count: 623 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.169.828 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.171.532 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 724 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.183.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 156, total running count: 624 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.185.233 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.186.794 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 725 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.199.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 157, total running count: 625 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.200.727 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.202.358 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 726 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.215.262 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 158, total running count: 626 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.216.533 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.218.941 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 727 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.230.858 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 159, total running count: 627 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.232.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.234.547 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 728 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.246.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 160, total running count: 628 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.247.832 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.249.891 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 729 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.262.295 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 161, total running count: 629 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.263.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.265.307 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 730 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.277.945 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 162, total running count: 630 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.278.898 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.280.995 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 731 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.293.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 163, total running count: 631 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.294.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.296.916 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 732 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.309.036 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 164, total running count: 632 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.310.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.312.514 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 733 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.324.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 165, total running count: 633 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.326.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.328.025 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 734 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.340.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 166, total running count: 634 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.341.679 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.343.824 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 735 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.355.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 167, total running count: 635 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.357.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.359.675 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 736 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.371.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 168, total running count: 636 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.372.900 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.375.329 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 737 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.386.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 169, total running count: 637 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.388.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.390.999 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 738 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.402.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 170, total running count: 638 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.404.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.406.980 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 739 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.418.876 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 171, total running count: 639 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.420.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.422.964 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 740 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.434.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 172, total running count: 640 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.435.851 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.437.740 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 741 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.450.028 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 173, total running count: 641 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.451.715 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.454.297 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 742 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.466.011 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 174, total running count: 642 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.467.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.469.901 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 743 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.482.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 175, total running count: 643 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.483.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.485.367 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 744 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.497.758 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 176, total running count: 644 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.498.809 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.501.016 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 745 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.513.154 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 177, total running count: 645 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.514.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.516.921 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 746 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.528.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 178, total running count: 646 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.529.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.531.398 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 747 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.544.241 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 179, total running count: 647 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.545.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.548.131 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 748 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.559.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 180, total running count: 648 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.560.949 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.563.435 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 749 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.575.301 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 181, total running count: 649 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.576.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.578.943 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 750 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.590.863 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 182, total running count: 650 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.591.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.594.519 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 751 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.606.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 183, total running count: 651 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.607.818 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.610.127 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 752 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.622.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 184, total running count: 652 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.623.375 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.625.786 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 753 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.637.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 185, total running count: 653 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.639.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.641.625 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 754 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.653.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 186, total running count: 654 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.654.916 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.656.566 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 755 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.669.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 187, total running count: 655 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.670.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.673.408 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 756 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.685.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 188, total running count: 656 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.686.329 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.687.983 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 757 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.700.580 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 189, total running count: 657 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.702.109 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.704.040 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 758 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.716.338 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 190, total running count: 658 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.717.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.719.381 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 759 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.731.878 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 191, total running count: 659 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.733.059 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.734.806 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 760 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.747.522 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 192, total running count: 660 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.748.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.750.217 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 761 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.762.992 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 193, total running count: 661 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.764.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.766.686 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 762 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.778.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 194, total running count: 662 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.779.962 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.782.287 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 763 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.794.771 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 195, total running count: 663 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.795.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.797.759 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 764 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.810.218 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 196, total running count: 664 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.811.302 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.813.052 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 765 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.825.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 197, total running count: 665 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.827.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.829.751 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 766 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.841.263 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 198, total running count: 666 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.842.562 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.845.248 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 767 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.857.353 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 199, total running count: 667 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.858.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.860.885 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 768 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.873.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 200, total running count: 668 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.873.997 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.876.478 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 769 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.888.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 201, total running count: 669 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.889.837 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.892.233 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 770 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.904.164 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 202, total running count: 670 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.905.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.906.812 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 771 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.920.016 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 203, total running count: 671 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.921.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.923.528 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 772 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:37.935.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 204, total running count: 672 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.936.878 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.938.907 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 773 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.951.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 205, total running count: 673 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:37.952.404 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.954.371 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 774 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.966.731 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 206, total running count: 674 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:37.968.056 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.970.154 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 775 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.982.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 207, total running count: 675 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:37.983.680 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:37.986.157 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 776 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.998.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 208, total running count: 676 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:37.999.132 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.001.762 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 777 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:38.013.408 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 209, total running count: 677 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.014.936 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.017.298 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 778 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.029.350 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 210, total running count: 678 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:38.030.670 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.032.987 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 779 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.045.024 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 211, total running count: 679 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:38.046.319 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.048.735 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 780 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.060.793 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 212, total running count: 680 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.061.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.064.385 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 781 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.076.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 213, total running count: 681 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:38.077.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.078.853 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 782 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.091.545 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 214, total running count: 682 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:38.092.969 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.095.375 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 783 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.107.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 215, total running count: 683 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.108.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.110.969 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 784 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.123.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 216, total running count: 684 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.124.348 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.126.378 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 785 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.138.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 217, total running count: 685 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.140.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.141.741 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 786 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.154.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 218, total running count: 686 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.155.547 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.158.238 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 787 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.170.139 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 219, total running count: 687 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.171.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.173.800 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 788 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:38.185.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 220, total running count: 688 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.186.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.189.401 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 789 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:38.200.948 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 221, total running count: 689 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:38.202.424 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.203.929 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 790 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.216.716 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 222, total running count: 690 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:38.218.035 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.219.759 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 791 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.232.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 223, total running count: 691 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.233.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.235.288 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 792 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:38.247.944 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 224, total running count: 692 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.249.120 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.250.736 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 793 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.263.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 225, total running count: 693 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.264.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.266.306 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 794 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.278.987 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 226, total running count: 694 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:38.280.015 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.281.791 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 795 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.294.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 227, total running count: 695 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.295.693 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.297.398 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 796 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.309.905 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 228, total running count: 696 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:38.311.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.313.262 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 797 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.325.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 229, total running count: 697 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.326.641 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.328.962 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 798 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:38.341.084 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 230, total running count: 698 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.342.346 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.344.807 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 799 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.356.843 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 231, total running count: 699 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.357.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.360.624 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 800 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:38.372.086 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 232, total running count: 700 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:38.373.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.374.928 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 801 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.387.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 233, total running count: 701 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.388.888 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.391.476 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 802 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:38.403.002 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 234, total running count: 702 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.404.572 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.406.928 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 803 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.418.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 235, total running count: 703 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.419.989 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.422.502 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 804 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:38.434.292 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 236, total running count: 704 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.435.412 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.437.894 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 805 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:38.449.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 237, total running count: 705 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.450.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.452.469 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 806 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.465.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 238, total running count: 706 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:38.466.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.467.931 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 807 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.480.589 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 239, total running count: 707 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.481.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.483.784 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 808 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.496.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 240, total running count: 708 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.497.381 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.499.374 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 809 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:38.511.629 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 241, total running count: 709 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.512.948 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.514.852 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 810 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.527.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 242, total running count: 710 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.528.281 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.530.171 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 811 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:38.542.615 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 243, total running count: 711 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.544.019 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.546.652 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 812 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.558.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 244, total running count: 712 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.559.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.562.216 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 813 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:38.573.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 245, total running count: 713 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.575.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.577.767 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 814 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.589.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 246, total running count: 714 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.590.936 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.593.259 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 815 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.605.248 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 247, total running count: 715 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:38.606.681 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.608.542 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 816 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.621.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 248, total running count: 716 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.622.304 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.624.101 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 817 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:38.636.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 249, total running count: 717 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.637.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.640.506 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 818 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.652.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 250, total running count: 718 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:38.653.720 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.655.972 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 819 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.668.036 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 251, total running count: 719 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:38.669.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.671.393 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 820 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.683.498 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 252, total running count: 720 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.684.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.686.782 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 821 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.699.345 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 253, total running count: 721 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.700.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.702.223 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 822 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.714.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 254, total running count: 722 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.715.861 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.717.762 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 823 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.730.116 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 255, total running count: 723 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.731.308 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.733.162 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 824 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.745.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 256, total running count: 724 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:38.746.642 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.748.636 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 825 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.760.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 257, total running count: 725 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:38.762.130 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.763.986 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 826 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.776.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 258, total running count: 726 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.777.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.779.303 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 827 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:38.791.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 259, total running count: 727 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.793.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.794.832 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 828 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:38.807.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 260, total running count: 728 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:38.808.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.810.308 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 829 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.822.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 261, total running count: 729 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:38.823.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.825.600 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 830 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.838.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 262, total running count: 730 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.839.385 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.841.031 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 831 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.853.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 263, total running count: 731 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.855.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.857.593 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 832 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:38.869.326 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 264, total running count: 732 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.870.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.873.086 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 833 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.884.829 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 265, total running count: 733 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:38.886.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.888.475 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 834 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:38.900.255 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 266, total running count: 734 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.901.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.903.927 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 835 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:38.915.811 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 267, total running count: 735 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.917.128 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.919.335 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 836 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.931.627 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 268, total running count: 736 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.932.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.934.837 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 837 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:38.946.872 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 269, total running count: 737 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.948.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.950.171 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 838 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.962.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 270, total running count: 738 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.963.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.965.751 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 839 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:38.978.148 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 271, total running count: 739 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.979.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.981.043 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 840 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:38.993.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 272, total running count: 740 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:38.994.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:38.996.500 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 841 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.009.136 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 273, total running count: 741 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.010.613 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.013.227 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 842 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.025.139 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 274, total running count: 742 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.026.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.028.672 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 843 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.040.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 275, total running count: 743 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.041.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.044.285 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 844 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.056.056 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 276, total running count: 744 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.057.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.058.961 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 845 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.071.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 277, total running count: 745 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.072.871 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.074.881 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 846 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.087.433 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 278, total running count: 746 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.088.623 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.090.554 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 847 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.103.162 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 279, total running count: 747 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.104.107 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.106.412 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 848 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.118.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 280, total running count: 748 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.119.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.122.168 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 849 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.133.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 281, total running count: 749 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.135.277 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.137.757 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 850 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.149.722 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 282, total running count: 750 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.150.756 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.153.342 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 851 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.165.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 283, total running count: 751 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.166.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.167.861 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 852 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.180.677 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 284, total running count: 752 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.181.649 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.183.433 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 853 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.195.955 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 285, total running count: 753 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.197.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.199.272 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 854 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.211.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 286, total running count: 754 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.212.508 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.214.856 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 855 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.227.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 287, total running count: 755 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.228.053 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.230.234 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 856 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.242.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 288, total running count: 756 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.243.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.245.596 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 857 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.258.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 289, total running count: 757 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.259.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.262.292 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 858 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.274.037 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 290, total running count: 758 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.275.149 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.277.875 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 859 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.289.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 291, total running count: 759 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.290.792 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.292.372 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 860 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.305.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 292, total running count: 760 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.306.400 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.308.248 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 861 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.320.641 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 293, total running count: 761 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.321.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.324.003 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 862 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.336.127 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 294, total running count: 762 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.337.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.339.514 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 863 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.351.738 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 295, total running count: 763 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.352.903 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.354.948 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 864 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.367.252 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 296, total running count: 764 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.368.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.370.384 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 865 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.382.666 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 297, total running count: 765 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.383.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.385.679 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 866 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.398.375 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 298, total running count: 766 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.399.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.401.232 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 867 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.413.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 299, total running count: 767 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.414.968 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.416.556 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 868 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.429.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 300, total running count: 768 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.430.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.432.214 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 869 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.444.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 301, total running count: 769 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.445.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.447.903 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 870 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.460.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 302, total running count: 770 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.461.531 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.463.790 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 871 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.475.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 303, total running count: 771 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.477.073 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.479.498 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 872 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.491.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 304, total running count: 772 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.492.757 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.495.274 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 873 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.507.071 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 305, total running count: 773 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.508.208 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.509.896 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 874 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.522.782 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 306, total running count: 774 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.523.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.525.738 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 875 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.538.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 307, total running count: 775 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.539.318 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.541.266 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 876 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.553.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 308, total running count: 776 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.554.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.556.764 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 877 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.569.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 309, total running count: 777 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.570.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.572.025 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 878 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.584.758 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 310, total running count: 778 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.585.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.587.506 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 879 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.600.200 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 311, total running count: 779 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.601.498 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.603.862 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 880 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.615.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 312, total running count: 780 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.616.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.619.371 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 881 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.631.371 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 313, total running count: 781 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.632.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.634.853 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 882 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.647.182 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 314, total running count: 782 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.648.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.650.281 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 883 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.662.675 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 315, total running count: 783 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.663.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.665.775 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 884 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.678.251 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 316, total running count: 784 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.679.303 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.681.410 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 885 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.693.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 317, total running count: 785 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.694.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.696.948 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 886 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.709.191 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 318, total running count: 786 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.710.303 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.712.931 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 887 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.724.735 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 319, total running count: 787 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.725.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.727.359 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 888 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.739.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 320, total running count: 788 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.741.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.742.738 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 889 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.755.385 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 321, total running count: 789 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.756.601 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.758.213 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 890 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.770.988 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 322, total running count: 790 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.772.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.773.740 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 891 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.786.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 323, total running count: 791 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.787.447 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.789.301 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 892 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.801.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 324, total running count: 792 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.803.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.805.681 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 893 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.817.437 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 325, total running count: 793 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.818.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.821.021 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 894 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.832.841 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 326, total running count: 794 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.834.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.836.007 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 895 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.849.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 327, total running count: 795 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.850.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.852.110 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 896 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.864.872 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 328, total running count: 796 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.865.912 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.868.031 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 897 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.880.413 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 329, total running count: 797 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.881.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.883.924 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 898 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.895.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 330, total running count: 798 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.897.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.900.267 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 899 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.912.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 331, total running count: 799 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.913.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.915.351 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 900 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.928.371 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 332, total running count: 800 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.929.030 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.931.471 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 901 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.944.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 333, total running count: 801 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:39.944.759 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.947.319 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 902 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:39.959.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 334, total running count: 802 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.960.551 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.963.152 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 903 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.975.749 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 335, total running count: 803 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:39.976.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.978.065 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 904 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:39.991.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 336, total running count: 804 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:39.992.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:39.994.355 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 905 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.007.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 337, total running count: 805 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:40.008.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.010.214 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 906 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.023.278 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 338, total running count: 806 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:40.023.790 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.026.283 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 907 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.038.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 339, total running count: 807 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.039.645 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.041.781 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 908 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.054.829 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 340, total running count: 808 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.055.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.057.955 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 909 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.070.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 341, total running count: 809 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.070.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.072.695 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 910 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.086.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 342, total running count: 810 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.086.759 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.088.595 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 911 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.101.762 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 343, total running count: 811 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.102.508 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.104.390 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 912 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.117.617 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 344, total running count: 812 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.118.325 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.120.036 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 913 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.133.462 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 345, total running count: 813 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.134.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.137.023 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 914 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:40.149.559 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 346, total running count: 814 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.150.314 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.152.953 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 915 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.165.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 347, total running count: 815 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.166.128 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.168.115 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 916 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.181.208 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 348, total running count: 816 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.181.665 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.183.398 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 917 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.196.746 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 349, total running count: 817 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.197.145 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.199.576 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 918 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:40.212.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 350, total running count: 818 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:40.212.973 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.215.315 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 919 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.227.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 351, total running count: 819 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.228.515 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.230.933 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 920 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.243.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 352, total running count: 820 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.244.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.246.682 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 921 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.259.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 353, total running count: 821 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:40.259.778 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.262.201 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 922 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.274.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 354, total running count: 822 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.275.442 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.278.104 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 923 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.290.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 355, total running count: 823 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.290.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.292.652 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 924 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.305.738 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 356, total running count: 824 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.306.567 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.308.181 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 925 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:40.321.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 357, total running count: 825 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.322.054 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.323.885 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 926 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.337.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 358, total running count: 826 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.337.765 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.339.814 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 927 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.352.545 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 359, total running count: 827 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.353.308 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.355.331 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 928 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.368.314 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 360, total running count: 828 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.369.068 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.370.897 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 929 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.384.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 361, total running count: 829 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.384.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.386.564 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 930 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.399.443 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 362, total running count: 830 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.400.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.402.332 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 931 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.415.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 363, total running count: 831 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.415.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.418.178 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 932 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.430.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 364, total running count: 832 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.430.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.432.717 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 933 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.445.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 365, total running count: 833 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.446.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.448.518 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 934 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.461.616 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 366, total running count: 834 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.462.089 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.463.914 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 935 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.477.155 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 367, total running count: 835 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.477.501 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.479.553 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 936 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.492.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 368, total running count: 836 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.492.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.495.797 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 937 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.507.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 369, total running count: 837 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.508.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.510.779 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 938 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.523.552 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 370, total running count: 838 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.524.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.526.294 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 939 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.539.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 371, total running count: 839 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.539.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.541.911 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 940 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.554.503 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 372, total running count: 840 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.555.275 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.557.423 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 941 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.570.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 373, total running count: 841 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.571.035 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.573.062 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 942 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.585.930 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 374, total running count: 842 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:40.586.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.589.517 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 943 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.601.481 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 375, total running count: 843 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.602.348 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.604.100 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 944 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.617.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 376, total running count: 844 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.617.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.620.188 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 945 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.632.832 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 377, total running count: 845 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.633.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.635.838 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 946 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.648.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 378, total running count: 846 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.649.051 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.651.772 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 947 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:40.663.811 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 379, total running count: 847 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.664.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.667.584 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 948 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.680.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 380, total running count: 848 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.680.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.682.172 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 949 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:40.695.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 381, total running count: 849 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.696.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.697.863 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 950 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.710.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 382, total running count: 850 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.711.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.713.797 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 951 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.726.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 383, total running count: 851 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.727.601 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.729.612 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 952 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:40.742.523 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 384, total running count: 852 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.743.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.745.584 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 953 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.758.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 385, total running count: 853 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:40.758.780 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.761.489 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 954 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.773.716 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 386, total running count: 854 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.774.501 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.776.224 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 955 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.789.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 387, total running count: 855 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.790.037 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.791.934 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 956 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.804.900 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 388, total running count: 856 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.805.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.807.671 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 957 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:40.820.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 389, total running count: 857 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.821.097 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.823.611 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 958 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:40.836.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 390, total running count: 858 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:40.836.969 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.839.520 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 959 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:40.851.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 391, total running count: 859 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.852.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.854.115 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 960 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:40.867.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 392, total running count: 860 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:40.868.059 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.869.659 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 961 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.883.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 393, total running count: 861 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.883.716 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.885.356 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 962 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.898.771 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 394, total running count: 862 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.899.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.901.385 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 963 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.914.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 395, total running count: 863 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.914.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.917.447 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 964 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.929.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 396, total running count: 864 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.930.264 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.932.155 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 965 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:40.945.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 397, total running count: 865 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.945.964 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.947.753 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 966 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.960.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 398, total running count: 866 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:40.961.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.963.494 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 967 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.976.535 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 399, total running count: 867 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.977.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.978.940 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 968 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:40.992.219 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 400, total running count: 868 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:40.992.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:40.994.750 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 969 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.007.486 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 401, total running count: 869 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:41.008.314 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.010.659 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 970 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.023.177 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 402, total running count: 870 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.023.848 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.026.255 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 971 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.038.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 403, total running count: 871 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.039.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.041.847 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 972 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.054.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 404, total running count: 872 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.055.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.057.325 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 973 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.069.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 405, total running count: 873 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.070.719 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.073.034 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 974 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.085.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 406, total running count: 874 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.086.311 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.088.389 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 975 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.101.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 407, total running count: 875 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.102.043 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.103.962 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 976 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.117.046 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 408, total running count: 876 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.117.441 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.119.434 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 977 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:41.132.427 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 409, total running count: 877 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.133.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.134.926 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 978 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.148.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 410, total running count: 878 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.148.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.150.835 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 979 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.163.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 411, total running count: 879 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.164.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.166.674 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 980 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:41.178.877 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 412, total running count: 880 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.179.552 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.181.417 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 981 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:41.194.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 413, total running count: 881 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.195.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.197.435 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 982 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.210.115 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 414, total running count: 882 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.210.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.213.094 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 983 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.225.303 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 415, total running count: 883 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.226.002 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.227.666 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 984 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.240.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 416, total running count: 884 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.241.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.244.612 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 985 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.256.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 417, total running count: 885 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.257.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.259.160 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 986 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.272.275 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 418, total running count: 886 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.272.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.275.046 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 987 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.287.782 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 419, total running count: 887 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.288.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.291.020 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 988 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.303.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 420, total running count: 888 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:41.303.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.305.788 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 989 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.318.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 421, total running count: 889 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.319.416 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.321.574 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 990 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.334.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 422, total running count: 890 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.335.444 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.337.537 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 991 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.350.324 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 423, total running count: 891 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.350.951 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.353.322 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 992 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.365.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 424, total running count: 892 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.366.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.369.319 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 993 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.381.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 425, total running count: 893 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.382.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.383.939 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 994 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.397.073 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 426, total running count: 894 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.397.782 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.399.582 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 995 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.412.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 427, total running count: 895 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.413.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.415.023 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 996 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:41.428.155 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 428, total running count: 896 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.428.752 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.430.907 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 997 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.443.670 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 429, total running count: 897 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.444.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.445.791 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 998 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.459.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 430, total running count: 898 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.459.783 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.461.454 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 999 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.474.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 431, total running count: 899 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:41.475.176 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.477.363 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1000 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:41.490.089 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 432, total running count: 900 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.490.642 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.493.308 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1001 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.505.432 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 433, total running count: 901 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.506.146 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.508.131 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1002 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.520.987 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 434, total running count: 902 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.521.889 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.524.078 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1003 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.536.701 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 435, total running count: 903 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.537.543 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.539.734 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1004 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.552.624 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 436, total running count: 904 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.553.066 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.555.435 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1005 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:41.567.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 437, total running count: 905 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.568.689 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.571.171 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1006 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.583.978 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 438, total running count: 906 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.584.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.586.826 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1007 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.599.352 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 439, total running count: 907 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.599.856 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.602.584 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1008 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.614.785 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 440, total running count: 908 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.615.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.617.556 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1009 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.630.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 441, total running count: 909 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.630.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.633.344 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1010 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.646.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 442, total running count: 910 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.646.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.649.254 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1011 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:41.661.324 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 443, total running count: 911 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.662.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.663.918 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1012 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.676.874 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 444, total running count: 912 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:41.677.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.679.593 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1013 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.692.547 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 445, total running count: 913 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.693.177 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.695.450 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1014 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:41.707.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 446, total running count: 914 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.708.813 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.711.425 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1015 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.723.727 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 447, total running count: 915 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.724.444 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.726.174 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1016 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:41.739.370 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 448, total running count: 916 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.739.930 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.741.777 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1017 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.754.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 449, total running count: 917 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.755.567 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.757.663 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1018 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.770.724 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 450, total running count: 918 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.771.073 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.773.796 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1019 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:41.785.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 451, total running count: 919 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.786.567 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.788.238 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1020 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.801.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 452, total running count: 920 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.802.148 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.803.966 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1021 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.817.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 453, total running count: 921 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:41.817.897 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.819.773 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1022 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.832.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 454, total running count: 922 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.833.532 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.835.786 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1023 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.848.433 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 455, total running count: 923 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.849.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.851.525 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1024 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.864.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 456, total running count: 924 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.864.961 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.867.273 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1025 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.879.960 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 457, total running count: 925 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.880.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.883.317 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1026 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.895.516 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 458, total running count: 926 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.896.165 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.897.891 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1027 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.911.097 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 459, total running count: 927 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:41.911.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.913.826 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1028 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.926.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 460, total running count: 928 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.927.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.929.663 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1029 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.942.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 461, total running count: 929 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.942.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.945.495 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1030 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.957.748 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 462, total running count: 930 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.958.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.961.012 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1031 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:41.973.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 463, total running count: 931 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:41.974.248 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.976.907 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1032 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:41.989.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 464, total running count: 932 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:41.989.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:41.991.506 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1033 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.004.898 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 465, total running count: 933 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.005.317 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.007.265 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1034 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.020.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 466, total running count: 934 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.020.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.022.910 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1035 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.035.926 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 467, total running count: 935 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.036.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.038.604 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1036 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.051.510 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 468, total running count: 936 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:40:42.052.425 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_1 execution count: 2, execution time: 7280.03 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:40:42.052.589 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_1 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:40:42.052.704 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty epoch: 2 step: 468, loss is 2.3046505451202393 [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:40:42.053.460 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.h:76] ContinueSend] continue send at the beginning of the epoch [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:40:42.054.377 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:40:42.054.435 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:40:42.054.477 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_1 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.054.778 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.056.746 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1037 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.069.752 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 1, total running count: 937 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.070.988 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.072.776 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1038 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.086.061 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 2, total running count: 938 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.087.007 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.089.711 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1039 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.101.722 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 3, total running count: 939 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.102.560 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.104.132 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1040 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.117.465 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 4, total running count: 940 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.118.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.119.844 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1041 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.133.324 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 5, total running count: 941 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.133.850 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.135.710 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1042 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.148.900 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 6, total running count: 942 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.149.388 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.151.568 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1043 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.164.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 7, total running count: 943 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.164.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.167.394 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1044 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.179.589 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 8, total running count: 944 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.180.465 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.183.175 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1045 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.195.243 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 9, total running count: 945 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.195.943 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.197.897 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1046 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.211.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 10, total running count: 946 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.211.468 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.213.729 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1047 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.226.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 11, total running count: 947 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.227.039 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.229.481 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1048 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.241.925 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 12, total running count: 948 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.242.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.244.253 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1049 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.257.754 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 13, total running count: 949 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.258.584 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.261.455 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1050 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.273.404 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 14, total running count: 950 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.274.278 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.276.736 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1051 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.289.054 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 15, total running count: 951 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.289.665 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.292.248 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1052 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.304.620 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 16, total running count: 952 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.305.187 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.306.781 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1053 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.320.073 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 17, total running count: 953 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.320.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.322.681 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1054 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.335.417 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 18, total running count: 954 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.336.011 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.338.245 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1055 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.350.859 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 19, total running count: 955 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.351.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.353.682 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1056 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.366.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 20, total running count: 956 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.367.068 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.369.216 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1057 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.381.775 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 21, total running count: 957 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.382.530 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.384.142 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1058 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.397.521 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 22, total running count: 958 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.398.028 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.399.865 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1059 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.413.077 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 23, total running count: 959 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.413.425 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.415.495 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1060 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.428.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 24, total running count: 960 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.429.089 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.430.888 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1061 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.443.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 25, total running count: 961 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.444.583 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.446.431 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1062 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.459.493 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 26, total running count: 962 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.460.259 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.462.237 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1063 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.475.247 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 27, total running count: 963 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.475.606 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.478.083 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1064 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.490.701 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 28, total running count: 964 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.491.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.492.672 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1065 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.505.848 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 29, total running count: 965 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.506.580 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.508.277 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1066 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.521.532 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 30, total running count: 966 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.522.178 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.524.086 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1067 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.537.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 31, total running count: 967 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.537.465 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.539.842 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1068 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.552.347 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 32, total running count: 968 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.552.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.555.342 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1069 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.567.769 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 33, total running count: 969 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.568.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.570.817 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1070 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.583.514 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 34, total running count: 970 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.583.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.586.440 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1071 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.598.601 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 35, total running count: 971 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.599.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.601.807 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1072 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.614.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 36, total running count: 972 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.614.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.617.376 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1073 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.629.501 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 37, total running count: 973 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.630.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.631.973 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1074 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.645.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 38, total running count: 974 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.645.864 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.647.826 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1075 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.660.728 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 39, total running count: 975 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.661.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.663.390 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1076 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.675.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 40, total running count: 976 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.676.756 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.678.960 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1077 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.691.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 41, total running count: 977 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.692.187 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.694.351 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1078 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.707.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 42, total running count: 978 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.707.694 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.709.809 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1079 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.722.460 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 43, total running count: 979 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.723.247 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.725.344 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1080 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.738.102 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 44, total running count: 980 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.738.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.741.026 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1081 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.753.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 45, total running count: 981 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.754.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.756.661 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1082 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.769.199 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 46, total running count: 982 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.769.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.772.032 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1083 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.784.785 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 47, total running count: 983 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.785.156 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.787.767 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1084 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.799.839 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 48, total running count: 984 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.800.686 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.803.406 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1085 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.815.606 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 49, total running count: 985 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.815.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.817.619 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1086 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.830.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 50, total running count: 986 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.831.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.832.977 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1087 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.846.135 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 51, total running count: 987 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.846.627 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.848.735 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1088 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.861.442 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 52, total running count: 988 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.862.163 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.864.336 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1089 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.876.749 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 53, total running count: 989 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.877.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.880.254 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1090 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.892.097 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 54, total running count: 990 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.892.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.894.580 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1091 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.907.624 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 55, total running count: 991 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.908.523 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.911.133 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1092 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.923.556 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 56, total running count: 992 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:42.924.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.926.697 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1093 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.939.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 57, total running count: 993 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:42.939.692 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.942.228 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1094 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.954.381 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 58, total running count: 994 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.955.200 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.957.783 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1095 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.969.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 59, total running count: 995 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:42.970.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.973.117 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1096 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:42.985.235 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 60, total running count: 996 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:42.985.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:42.988.527 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1097 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.000.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 61, total running count: 997 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.001.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.003.855 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1098 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:43.016.543 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 62, total running count: 998 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.017.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.019.301 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1099 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:43.032.285 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 63, total running count: 999 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.033.131 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.035.745 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1100 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.047.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 64, total running count: 1000 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:43.048.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.050.070 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1101 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.063.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 65, total running count: 1001 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.063.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.065.811 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1102 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.078.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 66, total running count: 1002 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.079.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.081.662 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1103 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.094.176 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 67, total running count: 1003 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.094.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.096.113 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1104 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.109.343 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 68, total running count: 1004 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.109.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.111.519 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1105 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.124.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 69, total running count: 1005 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.125.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.126.891 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1106 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.140.313 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 70, total running count: 1006 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:43.141.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.143.399 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1107 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.155.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 71, total running count: 1007 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.156.616 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.158.765 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1108 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.171.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 72, total running count: 1008 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:43.172.111 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.174.251 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1109 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.186.766 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 73, total running count: 1009 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.187.421 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.189.605 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1110 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.202.286 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 74, total running count: 1010 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.202.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.204.956 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1111 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.217.661 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 75, total running count: 1011 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:43.218.495 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.220.489 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1112 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.233.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 76, total running count: 1012 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.233.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.235.995 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1113 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.248.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 77, total running count: 1013 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.249.580 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.251.490 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1114 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.264.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 78, total running count: 1014 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.265.024 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.267.307 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1115 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.279.933 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 79, total running count: 1015 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.280.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.282.787 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1116 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.295.478 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 80, total running count: 1016 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.295.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.298.498 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1117 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.310.876 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 81, total running count: 1017 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.311.412 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.313.848 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1118 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.326.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 82, total running count: 1018 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.327.146 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.329.075 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1119 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.342.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 83, total running count: 1019 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.342.584 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.344.533 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1120 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.357.380 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 84, total running count: 1020 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.358.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.360.146 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1121 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.372.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 85, total running count: 1021 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.373.200 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.375.768 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1122 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.388.265 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 86, total running count: 1022 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.388.798 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.390.368 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1123 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.403.489 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 87, total running count: 1023 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.404.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.406.151 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1124 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.419.311 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 88, total running count: 1024 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.420.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.422.026 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1125 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.435.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 89, total running count: 1025 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:43.435.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.437.944 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1126 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.450.346 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 90, total running count: 1026 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.451.150 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.453.617 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1127 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.465.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 91, total running count: 1027 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.466.641 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.469.265 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1128 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.481.533 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 92, total running count: 1028 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:43.482.079 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.483.784 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1129 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.497.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 93, total running count: 1029 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.497.645 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.499.469 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1130 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.514.120 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 94, total running count: 1030 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.514.488 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.517.138 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1131 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:43.530.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 95, total running count: 1031 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.530.666 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.532.608 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1132 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.545.780 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 96, total running count: 1032 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.546.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.548.008 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1133 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.561.028 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 97, total running count: 1033 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:43.561.352 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.563.480 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1134 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.576.290 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 98, total running count: 1034 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.576.641 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.579.070 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1135 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.591.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 99, total running count: 1035 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.592.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.594.555 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1136 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:43.606.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 100, total running count: 1036 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.607.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.609.361 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1137 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.622.164 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 101, total running count: 1037 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.622.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.624.999 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1138 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.637.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 102, total running count: 1038 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.637.666 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.639.356 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1139 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:43.652.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 103, total running count: 1039 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.652.905 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.654.993 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1140 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.667.742 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 104, total running count: 1040 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.668.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.670.531 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1141 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:43.683.589 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 105, total running count: 1041 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:43.684.246 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.685.833 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1142 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:43.698.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 106, total running count: 1042 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.699.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.701.461 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1143 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.714.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 107, total running count: 1043 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.715.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.717.255 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1144 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.730.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 108, total running count: 1044 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.730.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.733.235 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1145 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.745.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 109, total running count: 1045 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.746.326 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.748.650 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1146 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.761.106 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 110, total running count: 1046 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.761.683 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.764.097 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1147 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.776.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 111, total running count: 1047 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:43.777.071 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.779.437 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1148 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.792.053 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 112, total running count: 1048 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.792.616 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.794.946 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1149 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.807.476 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 113, total running count: 1049 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:43.808.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.810.308 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1150 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:43.823.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 114, total running count: 1050 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.823.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.825.734 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1151 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.838.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 115, total running count: 1051 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.839.002 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.841.239 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1152 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.854.045 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 116, total running count: 1052 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.854.381 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.856.948 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1153 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.869.267 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 117, total running count: 1053 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.869.978 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.872.556 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1154 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.884.831 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 118, total running count: 1054 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.885.350 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.886.965 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1155 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.900.177 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 119, total running count: 1055 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.900.583 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.902.825 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1156 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:43.915.338 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 120, total running count: 1056 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.915.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.918.355 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1157 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.931.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 121, total running count: 1057 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.931.496 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.933.793 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1158 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.946.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 122, total running count: 1058 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.948.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.950.584 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1159 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.963.734 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 123, total running count: 1059 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:43.965.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.967.657 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1160 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.980.147 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 124, total running count: 1060 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:43.980.791 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.983.456 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1161 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:43.995.689 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 125, total running count: 1061 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:43.996.251 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:43.997.856 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1162 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.011.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 126, total running count: 1062 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.011.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.013.368 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1163 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.026.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 127, total running count: 1063 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.027.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.029.301 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1164 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:44.042.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 128, total running count: 1064 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.042.878 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.045.180 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1165 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.057.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 129, total running count: 1065 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.058.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.060.631 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1166 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.073.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 130, total running count: 1066 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:44.073.798 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.076.024 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1167 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:44.088.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 131, total running count: 1067 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:44.089.015 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.091.638 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1168 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:44.103.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 132, total running count: 1068 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.104.614 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.107.356 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1169 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:44.119.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 133, total running count: 1069 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.120.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.122.064 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1170 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:44.135.115 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 134, total running count: 1070 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.135.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.137.793 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1171 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.150.302 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 135, total running count: 1071 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.150.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.153.596 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1172 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.165.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 136, total running count: 1072 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:44.166.391 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.168.183 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1173 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.181.349 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 137, total running count: 1073 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.181.885 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.183.748 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1174 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.196.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 138, total running count: 1074 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.197.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.199.456 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1175 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:44.212.111 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 139, total running count: 1075 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.212.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.214.176 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1176 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.227.259 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 140, total running count: 1076 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.227.785 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.229.789 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1177 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.242.856 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 141, total running count: 1077 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.243.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.245.314 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1178 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.258.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 142, total running count: 1078 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.259.488 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.261.382 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1179 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.274.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 143, total running count: 1079 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.275.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.277.196 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1180 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:44.289.853 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 144, total running count: 1080 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.290.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.292.942 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1181 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.305.799 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 145, total running count: 1081 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.306.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.308.729 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1182 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.321.184 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 146, total running count: 1082 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.321.769 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.324.293 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1183 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.336.619 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 147, total running count: 1083 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.337.142 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.338.699 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1184 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.351.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 148, total running count: 1084 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.352.523 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.354.321 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1185 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:44.367.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 149, total running count: 1085 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.367.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.369.578 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1186 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.382.690 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 150, total running count: 1086 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.383.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.385.221 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1187 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.398.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 151, total running count: 1087 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.398.858 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.400.552 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1188 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:44.413.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 152, total running count: 1088 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.415.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.418.279 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1189 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.430.926 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 153, total running count: 1089 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.432.068 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.433.757 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1190 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.447.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 154, total running count: 1090 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.447.813 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.450.296 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1191 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.462.727 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 155, total running count: 1091 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.463.481 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.465.749 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1192 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.478.136 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 156, total running count: 1092 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.479.086 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.481.292 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1193 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.494.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 157, total running count: 1093 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.494.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.497.395 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1194 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.509.645 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 158, total running count: 1094 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.510.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.512.085 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1195 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.525.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 159, total running count: 1095 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.525.795 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.527.492 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1196 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:44.540.731 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 160, total running count: 1096 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:44.541.149 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.542.988 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1197 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.556.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 161, total running count: 1097 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.560.737 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.562.938 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1198 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.575.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 162, total running count: 1098 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:44.576.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.578.525 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1199 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.591.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 163, total running count: 1099 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.591.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.593.999 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1200 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.606.854 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 164, total running count: 1100 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.607.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.609.479 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1201 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.622.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 165, total running count: 1101 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.622.978 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.625.302 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1202 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.637.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 166, total running count: 1102 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:44.638.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.641.125 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1203 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.653.267 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 167, total running count: 1103 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:44.653.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.656.641 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1204 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.668.602 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 168, total running count: 1104 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.669.434 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.670.977 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1205 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.684.391 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 169, total running count: 1105 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.684.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.686.594 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1206 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.699.722 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 170, total running count: 1106 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.700.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.702.331 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1207 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.715.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 171, total running count: 1107 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.719.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.721.750 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1208 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:44.734.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 172, total running count: 1108 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.734.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.737.354 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1209 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.749.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 173, total running count: 1109 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.750.721 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.752.647 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1210 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.765.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 174, total running count: 1110 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.766.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.767.889 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1211 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.781.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 175, total running count: 1111 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.781.458 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.783.361 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1212 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.796.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 176, total running count: 1112 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.796.865 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.798.956 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1213 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.811.878 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 177, total running count: 1113 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.812.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.814.486 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1214 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.827.128 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 178, total running count: 1114 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.829.468 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.831.421 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1215 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.844.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 179, total running count: 1115 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.846.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.848.477 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1216 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.860.845 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 180, total running count: 1116 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.861.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.863.972 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1217 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.876.750 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 181, total running count: 1117 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.877.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.879.379 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1218 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.892.503 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 182, total running count: 1118 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.893.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.895.937 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1219 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.908.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 183, total running count: 1119 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.908.939 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.911.364 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1220 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.923.828 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 184, total running count: 1120 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.926.149 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.928.049 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1221 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.941.052 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 185, total running count: 1121 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:44.944.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.945.947 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1222 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.959.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 186, total running count: 1122 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:44.959.877 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.961.716 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1223 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:44.974.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 187, total running count: 1123 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:44.975.622 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.977.503 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1224 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.990.423 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 188, total running count: 1124 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:44.991.226 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:44.993.220 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1225 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.006.192 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 189, total running count: 1125 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.007.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.008.936 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1226 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.021.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 190, total running count: 1126 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.022.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.024.728 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1227 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.037.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 191, total running count: 1127 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.040.255 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.042.779 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1228 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.055.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 192, total running count: 1128 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.055.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.058.189 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1229 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.070.734 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 193, total running count: 1129 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.071.576 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.073.683 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1230 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.086.249 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 194, total running count: 1130 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:45.086.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.089.022 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1231 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.101.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 195, total running count: 1131 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.102.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.104.469 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1232 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:45.116.993 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 196, total running count: 1132 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.117.925 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.119.950 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1233 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.132.899 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 197, total running count: 1133 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.135.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.137.778 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1234 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:45.150.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 198, total running count: 1134 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.152.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.154.717 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1235 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.167.372 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 199, total running count: 1135 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.168.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.170.589 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1236 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.182.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 200, total running count: 1136 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.184.178 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.186.428 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1237 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.198.960 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 201, total running count: 1137 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.199.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.201.851 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1238 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.214.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 202, total running count: 1138 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.215.768 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.218.418 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1239 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.230.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 203, total running count: 1139 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:45.231.462 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.233.870 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1240 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:45.246.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 204, total running count: 1140 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.253.522 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.255.564 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1241 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.268.618 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 205, total running count: 1141 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:45.271.131 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.273.456 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1242 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:45.286.165 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 206, total running count: 1142 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:45.287.108 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.289.074 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1243 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.301.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 207, total running count: 1143 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:45.303.035 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.305.758 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1244 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:45.318.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 208, total running count: 1144 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.319.042 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.321.361 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1245 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.334.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 209, total running count: 1145 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:45.335.042 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.337.189 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1246 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.349.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 210, total running count: 1146 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.351.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.354.063 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1247 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:45.366.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 211, total running count: 1147 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.367.217 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.369.787 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1248 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.382.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 212, total running count: 1148 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.383.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.385.408 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1249 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.397.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 213, total running count: 1149 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.398.975 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.400.888 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1250 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.413.665 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 214, total running count: 1150 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.414.623 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.416.528 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1251 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.429.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 215, total running count: 1151 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:45.430.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.432.375 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1252 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.445.236 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 216, total running count: 1152 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:45.446.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.448.074 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1253 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:45.461.318 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 217, total running count: 1153 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.462.742 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.464.696 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1254 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:45.477.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 218, total running count: 1154 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.478.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.480.385 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1255 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.493.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 219, total running count: 1155 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.494.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.495.774 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1256 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.508.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 220, total running count: 1156 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.509.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.511.507 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1257 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.524.720 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 221, total running count: 1157 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:45.525.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.527.380 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1258 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.540.466 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 222, total running count: 1158 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:45.541.592 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.543.944 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1259 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.556.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 223, total running count: 1159 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.557.239 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.559.529 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1260 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.572.027 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 224, total running count: 1160 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.573.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.575.258 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1261 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.588.028 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 225, total running count: 1161 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.588.982 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.591.119 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1262 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:45.603.876 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 226, total running count: 1162 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.604.701 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.606.879 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1263 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.619.630 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 227, total running count: 1163 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.620.317 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.622.497 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1264 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:45.635.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 228, total running count: 1164 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:45.635.805 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.637.996 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1265 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:45.650.937 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 229, total running count: 1165 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.651.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.653.316 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1266 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.666.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 230, total running count: 1166 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.666.982 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.668.870 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1267 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:45.681.839 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 231, total running count: 1167 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.682.716 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.684.929 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1268 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:45.697.583 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 232, total running count: 1168 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.698.404 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.700.578 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1269 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.713.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 233, total running count: 1169 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.714.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.716.435 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1270 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.729.024 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 234, total running count: 1170 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.729.756 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.732.150 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1271 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.744.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 235, total running count: 1171 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:45.745.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.747.761 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1272 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.760.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 236, total running count: 1172 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.760.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.763.222 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1273 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.775.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 237, total running count: 1173 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:45.776.653 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.778.795 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1274 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:45.791.510 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 238, total running count: 1174 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.792.248 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.794.256 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1275 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.807.105 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 239, total running count: 1175 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.807.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.809.866 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1276 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:45.822.668 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 240, total running count: 1176 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:45.823.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.825.560 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1277 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:45.838.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 241, total running count: 1177 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.838.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.841.424 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1278 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.853.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 242, total running count: 1178 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.854.486 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.856.677 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1279 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.869.439 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 243, total running count: 1179 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.870.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.872.150 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1280 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.884.833 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 244, total running count: 1180 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.885.350 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.887.523 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1281 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.900.325 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 245, total running count: 1181 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:45.900.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.902.972 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1282 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.915.711 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 246, total running count: 1182 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.916.556 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.918.507 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1283 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.931.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 247, total running count: 1183 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.931.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.933.871 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1284 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:45.946.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 248, total running count: 1184 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:45.947.412 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.949.683 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1285 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:45.962.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 249, total running count: 1185 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:45.963.212 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.965.568 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1286 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.977.989 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 250, total running count: 1186 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:45.978.640 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.980.210 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1287 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:45.993.552 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 251, total running count: 1187 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:45.994.292 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:45.996.766 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1288 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.009.176 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 252, total running count: 1188 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.010.053 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.012.148 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1289 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.024.988 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 253, total running count: 1189 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.025.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.027.627 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1290 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.040.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 254, total running count: 1190 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.041.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.042.952 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1291 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.055.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 255, total running count: 1191 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.056.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.058.325 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1292 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.071.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 256, total running count: 1192 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.071.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.073.971 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1293 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.086.785 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 257, total running count: 1193 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.087.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.089.290 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1294 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.102.200 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 258, total running count: 1194 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.103.474 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.105.769 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1295 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.118.523 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 259, total running count: 1195 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.119.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.121.377 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1296 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.134.205 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 260, total running count: 1196 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.134.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.137.324 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1297 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.149.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 261, total running count: 1197 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.150.109 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.152.019 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1298 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.165.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 262, total running count: 1198 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.165.412 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.167.752 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1299 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.180.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 263, total running count: 1199 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.181.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.183.414 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1300 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.196.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 264, total running count: 1200 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.196.498 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.198.139 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1301 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.211.335 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 265, total running count: 1201 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.212.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.214.710 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1302 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.227.056 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 266, total running count: 1202 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.227.597 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.229.366 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1303 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.242.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 267, total running count: 1203 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.243.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.245.393 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1304 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.257.982 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 268, total running count: 1204 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.258.317 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.259.862 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1305 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.273.129 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 269, total running count: 1205 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.273.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.276.413 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1306 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.288.485 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 270, total running count: 1206 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.289.032 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.290.733 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1307 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.304.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 271, total running count: 1207 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.304.709 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.307.342 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1308 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.319.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 272, total running count: 1208 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.320.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.321.762 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1309 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.335.118 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 273, total running count: 1209 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.335.839 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.338.407 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1310 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.350.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 274, total running count: 1210 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.351.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.353.790 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1311 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.366.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 275, total running count: 1211 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.367.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.369.349 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1312 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.382.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 276, total running count: 1212 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.382.567 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.384.787 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1313 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.397.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 277, total running count: 1213 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.398.130 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.400.081 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1314 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.413.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 278, total running count: 1214 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.413.396 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.415.414 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1315 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.428.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 279, total running count: 1215 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.428.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.430.779 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1316 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.443.644 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 280, total running count: 1216 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.444.249 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.446.242 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1317 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.459.131 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 281, total running count: 1217 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.459.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.461.723 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1318 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.474.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 282, total running count: 1218 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.475.473 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.477.034 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1319 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.490.267 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 283, total running count: 1219 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.490.864 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.492.518 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1320 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.505.720 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 284, total running count: 1220 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.506.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.508.128 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1321 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.521.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 285, total running count: 1221 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.522.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.523.885 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1322 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.537.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 286, total running count: 1222 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.537.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.539.540 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1323 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.552.822 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 287, total running count: 1223 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.553.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.554.960 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1324 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.568.217 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 288, total running count: 1224 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.568.574 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.570.494 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1325 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.583.697 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 289, total running count: 1225 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.584.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.586.223 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1326 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.599.121 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 290, total running count: 1226 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.600.187 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.602.098 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1327 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.615.097 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 291, total running count: 1227 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.615.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.617.605 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1328 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.630.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 292, total running count: 1228 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.630.985 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.632.973 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1329 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.646.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 293, total running count: 1229 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.646.443 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.648.536 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1330 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.661.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 294, total running count: 1230 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.661.861 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.664.506 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1331 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.676.702 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 295, total running count: 1231 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.677.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.678.922 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1332 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.692.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 296, total running count: 1232 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.692.964 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.695.614 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1333 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.707.937 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 297, total running count: 1233 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.708.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.711.146 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1334 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.723.483 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 298, total running count: 1234 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.723.851 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.725.730 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1335 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.738.837 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 299, total running count: 1235 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.739.343 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.741.255 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1336 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.754.308 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 300, total running count: 1236 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.755.400 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.757.035 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1337 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.770.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 301, total running count: 1237 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.771.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.773.853 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1338 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.786.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 302, total running count: 1238 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.786.698 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.789.380 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1339 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.801.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 303, total running count: 1239 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.801.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.804.070 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1340 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.816.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 304, total running count: 1240 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.817.440 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.819.911 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1341 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.832.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 305, total running count: 1241 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.832.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.835.634 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1342 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.847.804 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 306, total running count: 1242 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.848.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.851.010 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1343 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.863.205 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 307, total running count: 1243 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.864.128 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.866.348 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1344 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.879.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 308, total running count: 1244 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.879.619 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.881.768 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1345 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.894.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 309, total running count: 1245 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.895.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.897.197 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1346 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.910.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 310, total running count: 1246 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.910.789 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.912.594 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1347 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.925.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 311, total running count: 1247 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.926.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.928.000 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1348 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:46.941.265 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 312, total running count: 1248 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.942.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.943.695 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1349 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.957.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 313, total running count: 1249 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:46.957.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.959.481 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1350 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.972.459 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 314, total running count: 1250 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:46.973.251 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.974.966 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1351 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:46.988.065 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 315, total running count: 1251 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:46.988.691 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:46.990.497 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1352 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.004.453 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 316, total running count: 1252 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:47.004.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.007.326 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1353 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:47.020.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 317, total running count: 1253 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.020.721 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.022.899 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1354 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:47.035.820 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 318, total running count: 1254 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:47.036.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.038.350 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1355 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.051.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 319, total running count: 1255 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:47.051.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.053.778 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1356 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.066.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 320, total running count: 1256 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:47.066.997 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.069.268 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1357 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.082.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 321, total running count: 1257 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.082.647 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.085.082 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1358 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.097.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 322, total running count: 1258 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.098.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.099.775 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1359 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.113.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 323, total running count: 1259 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:47.113.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.116.393 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1360 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:47.128.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 324, total running count: 1260 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.129.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.131.798 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1361 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.144.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 325, total running count: 1261 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:47.145.052 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.147.437 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1362 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:47.160.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 326, total running count: 1262 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.160.915 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.163.866 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1363 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.176.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 327, total running count: 1263 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.176.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.178.841 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1364 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.191.839 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 328, total running count: 1264 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:47.192.516 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.194.440 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1365 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.208.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 329, total running count: 1265 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.209.865 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.225.086 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 330, total running count: 1266 [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.225.536 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1366 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.225.805 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.228.521 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1367 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.241.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 331, total running count: 1267 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.241.897 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.243.757 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1368 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.257.264 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 332, total running count: 1268 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.257.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.260.147 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1369 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.273.036 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 333, total running count: 1269 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.273.458 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.275.213 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1370 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.288.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 334, total running count: 1270 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:47.289.441 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.291.116 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1371 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.304.736 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 335, total running count: 1271 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.305.182 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.306.894 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1372 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:47.320.482 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 336, total running count: 1272 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:47.320.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.322.836 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1373 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.336.105 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 337, total running count: 1273 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.336.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.338.792 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1374 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:47.351.742 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 338, total running count: 1274 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:47.352.116 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.354.956 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1375 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:47.367.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 339, total running count: 1275 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.367.890 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.369.735 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1376 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:47.383.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 340, total running count: 1276 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:47.383.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.385.511 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1377 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.398.649 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 341, total running count: 1277 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.399.174 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.401.362 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1378 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:47.414.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 342, total running count: 1278 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:47.414.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.417.268 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1379 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:47.430.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 343, total running count: 1279 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.430.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.433.043 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1380 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:47.446.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 344, total running count: 1280 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.446.489 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.448.968 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1381 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.461.614 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 345, total running count: 1281 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.462.056 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.463.776 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1382 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.477.155 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 346, total running count: 1282 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.478.071 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.479.685 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1383 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.493.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 347, total running count: 1283 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.493.798 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.495.507 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1384 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.509.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 348, total running count: 1284 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.509.749 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.511.673 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1385 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.525.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 349, total running count: 1285 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.525.674 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.527.626 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1386 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.543.146 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 350, total running count: 1286 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.543.527 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.545.519 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1387 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.558.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 351, total running count: 1287 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.558.872 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.561.392 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1388 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.573.841 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 352, total running count: 1288 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.574.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.576.725 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1389 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.588.718 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 353, total running count: 1289 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.589.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.591.000 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1390 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.603.968 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 354, total running count: 1290 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.604.344 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.606.640 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1391 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.619.177 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 355, total running count: 1291 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.619.877 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.622.219 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1392 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.634.582 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 356, total running count: 1292 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.635.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.637.583 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1393 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.650.251 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 357, total running count: 1293 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:47.651.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.653.337 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1394 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.665.878 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 358, total running count: 1294 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.667.695 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.670.029 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1395 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.682.148 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 359, total running count: 1295 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.683.765 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.685.640 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1396 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.698.527 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 360, total running count: 1296 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.699.459 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.701.211 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1397 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.713.973 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 361, total running count: 1297 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.714.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.716.849 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1398 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.729.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 362, total running count: 1298 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.730.416 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.732.437 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1399 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.744.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 363, total running count: 1299 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.746.053 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.747.946 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1400 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.760.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 364, total running count: 1300 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.761.889 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.763.543 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1401 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.776.338 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 365, total running count: 1301 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.776.964 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.779.303 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1402 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.792.473 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 366, total running count: 1302 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:47.792.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.795.003 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1403 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.807.750 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 367, total running count: 1303 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.808.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.809.721 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1404 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.822.619 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 368, total running count: 1304 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.823.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.825.279 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1405 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.838.225 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 369, total running count: 1305 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.839.146 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.841.068 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1406 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.853.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 370, total running count: 1306 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.854.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.856.612 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1407 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.869.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 371, total running count: 1307 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.870.580 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.872.394 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1408 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.885.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 372, total running count: 1308 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.886.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.887.997 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1409 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.900.964 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 373, total running count: 1309 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.902.181 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.904.690 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1410 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.916.702 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 374, total running count: 1310 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.917.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.920.437 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1411 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.932.511 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 375, total running count: 1311 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.933.264 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.935.263 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1412 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.947.973 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 376, total running count: 1312 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.948.933 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.950.898 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1413 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.963.720 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 377, total running count: 1313 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.964.562 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.966.336 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1414 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:47.979.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 378, total running count: 1314 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.980.151 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.981.798 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1415 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:47.994.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 379, total running count: 1315 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:47.995.694 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:47.998.278 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1416 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.010.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 380, total running count: 1316 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.011.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.013.274 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1417 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.026.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 381, total running count: 1317 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.029.790 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.031.597 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1418 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.044.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 382, total running count: 1318 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.046.326 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.048.474 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1419 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.060.764 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 383, total running count: 1319 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.061.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.063.989 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1420 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.076.889 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 384, total running count: 1320 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.077.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.079.442 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1421 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.091.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 385, total running count: 1321 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.092.697 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.094.964 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1422 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.107.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 386, total running count: 1322 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.108.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.110.350 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1423 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.122.702 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 387, total running count: 1323 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.123.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.125.762 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1424 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.138.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 388, total running count: 1324 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.139.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.141.522 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1425 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.153.903 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 389, total running count: 1325 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.155.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.157.033 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1426 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.169.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 390, total running count: 1326 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.171.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.172.773 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1427 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.185.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 391, total running count: 1327 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.186.400 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.188.396 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1428 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.201.087 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 392, total running count: 1328 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.202.171 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.204.091 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1429 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:48.216.584 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 393, total running count: 1329 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.217.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.219.895 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1430 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.232.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 394, total running count: 1330 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.233.375 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.235.687 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1431 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.248.032 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 395, total running count: 1331 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.249.291 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.251.505 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1432 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.264.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 396, total running count: 1332 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.264.961 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.267.358 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1433 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.279.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 397, total running count: 1333 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.280.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.283.087 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1434 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.295.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 398, total running count: 1334 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.296.112 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.297.814 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1435 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.310.799 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 399, total running count: 1335 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.311.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.313.636 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1436 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.326.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 400, total running count: 1336 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.327.350 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.329.366 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1437 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.341.785 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 401, total running count: 1337 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.342.960 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.345.381 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1438 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.357.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 402, total running count: 1338 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.358.776 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.361.333 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1439 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.373.033 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 403, total running count: 1339 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.374.489 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.376.855 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1440 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.389.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 404, total running count: 1340 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.390.400 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.392.440 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1441 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.404.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 405, total running count: 1341 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.405.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.408.173 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1442 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.420.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 406, total running count: 1342 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.421.308 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.424.056 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1443 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.435.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 407, total running count: 1343 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.436.877 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.438.744 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1444 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.451.951 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 408, total running count: 1344 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.452.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.454.225 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1445 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.467.212 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 409, total running count: 1345 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.467.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.469.778 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1446 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.482.544 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 410, total running count: 1346 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.483.550 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.485.406 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1447 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.498.134 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 411, total running count: 1347 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.499.140 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.501.228 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1448 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.513.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 412, total running count: 1348 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.514.683 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.517.288 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1449 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.529.131 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 413, total running count: 1349 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.530.325 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.532.865 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1450 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.544.898 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 414, total running count: 1350 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.545.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.548.498 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1451 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:48.560.262 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 415, total running count: 1351 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.561.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.562.958 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1452 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.575.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 416, total running count: 1352 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.577.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.578.881 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1453 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.591.703 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 417, total running count: 1353 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.592.742 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.594.751 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1454 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.607.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 418, total running count: 1354 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.608.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.610.379 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1455 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.622.739 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 419, total running count: 1355 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.623.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.626.209 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1456 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.638.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 420, total running count: 1356 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.639.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.642.198 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1457 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.654.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 421, total running count: 1357 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.655.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.657.850 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1458 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.669.762 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 422, total running count: 1358 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.670.586 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.672.374 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1459 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.685.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 423, total running count: 1359 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.686.303 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.688.318 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1460 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.700.717 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 424, total running count: 1360 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.701.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.704.351 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1461 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.716.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 425, total running count: 1361 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.717.443 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.720.227 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1462 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.731.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 426, total running count: 1362 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.732.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.734.566 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1463 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.747.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 427, total running count: 1363 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.748.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.750.185 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1464 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.763.693 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 428, total running count: 1364 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.764.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.765.752 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1465 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.779.688 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 429, total running count: 1365 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.780.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.782.508 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1466 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.796.556 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 430, total running count: 1366 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.796.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.799.736 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1467 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:48.812.388 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 431, total running count: 1367 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:48.812.848 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.815.544 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1468 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.827.944 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 432, total running count: 1368 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.828.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.830.139 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1469 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:48.843.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 433, total running count: 1369 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:48.843.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.845.800 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1470 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.858.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 434, total running count: 1370 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.859.024 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.861.283 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1471 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.873.637 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 435, total running count: 1371 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.874.009 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.875.628 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1472 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.888.860 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 436, total running count: 1372 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.889.247 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.891.463 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1473 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.903.588 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 437, total running count: 1373 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.904.890 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.907.406 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1474 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.919.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 438, total running count: 1374 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.920.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.923.004 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1475 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.934.776 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 439, total running count: 1375 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.936.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.937.592 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1476 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.950.563 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 440, total running count: 1376 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.951.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.953.791 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1477 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.967.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 441, total running count: 1377 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:48.967.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.969.363 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1478 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.982.350 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 442, total running count: 1378 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:48.982.688 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.984.972 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1479 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.997.402 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 443, total running count: 1379 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:48.997.998 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:48.999.795 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1480 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.012.481 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 444, total running count: 1380 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.013.569 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.015.687 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1481 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.027.854 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 445, total running count: 1381 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.028.998 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.031.353 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1482 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.044.269 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 446, total running count: 1382 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.044.615 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.046.833 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1483 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.059.488 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 447, total running count: 1383 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.059.872 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.062.380 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1484 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.074.479 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 448, total running count: 1384 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.075.132 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.076.991 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1485 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.089.400 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 449, total running count: 1385 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.090.588 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.092.899 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1486 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.105.071 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 450, total running count: 1386 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.106.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.108.671 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1487 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.120.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 451, total running count: 1387 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.121.501 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.123.315 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1488 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.136.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 452, total running count: 1388 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.137.127 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.139.135 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1489 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.151.620 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 453, total running count: 1389 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.152.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.154.714 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1490 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.166.848 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 454, total running count: 1390 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.167.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.170.292 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1491 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.182.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 455, total running count: 1391 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.183.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.185.805 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1492 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.198.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 456, total running count: 1392 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.198.599 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.200.265 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1493 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.213.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 457, total running count: 1393 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.213.969 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.216.248 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1494 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.228.751 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 458, total running count: 1394 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.229.496 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.232.150 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1495 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.243.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 459, total running count: 1395 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.245.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.246.825 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1496 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.262.195 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 460, total running count: 1396 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.262.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.264.822 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1497 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.277.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 461, total running count: 1397 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.277.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.279.498 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1498 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.291.988 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 462, total running count: 1398 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.292.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.294.205 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1499 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.306.616 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 463, total running count: 1399 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.307.827 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.309.925 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1500 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.322.281 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 464, total running count: 1400 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.323.496 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.325.516 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1501 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:49.338.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 465, total running count: 1401 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.339.011 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.341.465 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1502 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.353.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 466, total running count: 1402 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.354.350 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.356.049 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1503 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.368.758 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 467, total running count: 1403 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.369.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.371.783 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1504 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.384.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 468, total running count: 1404 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:40:49.385.733 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_1 execution count: 3, execution time: 7331.1 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:40:49.385.919 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_1 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:40:49.386.030 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty epoch: 3 step: 468, loss is 2.318190336227417 [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:40:49.387.062 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.h:76] ContinueSend] continue send at the beginning of the epoch [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:40:49.388.183 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:40:49.388.248 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:40:49.388.293 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_1 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.388.657 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.391.093 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1505 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.403.942 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 1, total running count: 1405 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.404.722 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.406.921 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1506 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.419.216 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 2, total running count: 1406 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.420.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.421.990 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1507 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.434.853 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 3, total running count: 1407 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.436.077 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.437.649 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1508 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.450.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 4, total running count: 1408 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.451.841 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.453.529 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1509 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.466.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 5, total running count: 1409 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.467.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.469.520 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1510 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.481.575 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 6, total running count: 1410 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.482.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.485.214 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1511 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.497.271 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 7, total running count: 1411 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:49.498.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.500.914 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1512 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.513.035 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 8, total running count: 1412 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.514.515 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.516.719 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1513 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.529.389 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 9, total running count: 1413 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.530.295 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.532.730 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1514 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.544.705 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 10, total running count: 1414 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.545.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.548.520 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1515 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.560.269 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 11, total running count: 1415 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.561.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.564.133 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1516 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.575.790 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 12, total running count: 1416 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.577.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.579.733 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1517 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.591.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 13, total running count: 1417 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.592.521 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.594.401 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1518 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.607.775 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 14, total running count: 1418 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.608.153 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.610.296 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1519 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.622.594 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 15, total running count: 1419 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.623.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.626.028 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1520 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.638.556 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 16, total running count: 1420 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.638.978 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.641.645 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1521 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.653.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 17, total running count: 1421 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.654.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.656.237 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1522 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.668.804 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 18, total running count: 1422 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.669.796 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.671.958 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1523 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.684.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 19, total running count: 1423 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.685.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.687.674 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1524 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.699.809 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 20, total running count: 1424 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.700.928 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.702.971 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1525 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.715.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 21, total running count: 1425 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.716.596 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.718.328 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1526 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.730.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 22, total running count: 1426 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.732.008 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.733.726 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1527 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.746.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 23, total running count: 1427 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.747.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.749.515 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1528 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.761.776 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 24, total running count: 1428 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.762.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.765.215 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1529 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.777.485 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 25, total running count: 1429 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.778.398 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.781.049 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1530 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.792.785 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 26, total running count: 1430 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.793.874 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.795.419 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1531 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.808.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 27, total running count: 1431 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:49.809.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.812.145 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1532 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.824.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 28, total running count: 1432 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.824.770 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.826.560 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1533 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.839.182 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 29, total running count: 1433 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.839.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.842.446 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1534 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.854.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 30, total running count: 1434 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.855.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.857.012 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1535 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.869.541 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 31, total running count: 1435 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.870.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.872.627 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1536 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.885.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 32, total running count: 1436 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.886.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.888.053 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1537 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.901.399 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 33, total running count: 1437 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.901.848 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.903.507 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1538 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.916.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 34, total running count: 1438 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.916.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.918.784 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1539 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.931.496 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 35, total running count: 1439 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.932.222 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.934.225 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1540 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.946.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 36, total running count: 1440 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.947.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.949.583 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1541 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:49.962.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 37, total running count: 1441 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:49.963.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.965.259 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1542 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.977.644 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 38, total running count: 1442 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.978.550 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.980.913 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1543 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.993.389 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 39, total running count: 1443 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:49.993.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:49.995.436 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1544 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.008.347 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 40, total running count: 1444 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.009.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.011.215 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1545 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.023.619 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 41, total running count: 1445 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.024.815 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.026.725 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1546 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.039.178 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 42, total running count: 1446 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.040.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.042.306 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1547 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.054.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 43, total running count: 1447 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.055.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.057.649 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1548 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.070.163 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 44, total running count: 1448 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.071.317 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.073.258 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1549 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.086.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 45, total running count: 1449 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.086.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.088.522 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1550 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.101.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 46, total running count: 1450 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.102.154 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.103.962 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1551 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.116.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 47, total running count: 1451 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.117.444 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.119.417 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1552 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.131.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 48, total running count: 1452 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.133.052 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.134.866 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1553 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.147.249 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 49, total running count: 1453 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.148.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.150.377 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1554 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.163.308 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 50, total running count: 1454 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.164.028 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.165.883 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1555 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.178.458 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 51, total running count: 1455 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.179.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.181.448 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1556 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.194.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 52, total running count: 1456 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.195.260 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.197.254 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1557 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.209.766 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 53, total running count: 1457 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.210.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.213.065 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1558 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.225.053 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 54, total running count: 1458 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.226.096 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.228.582 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1559 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.240.433 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 55, total running count: 1459 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.241.417 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.243.957 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1560 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.256.226 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 56, total running count: 1460 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.256.859 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.258.525 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1561 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.271.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 57, total running count: 1461 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.272.336 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.273.977 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1562 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.287.155 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 58, total running count: 1462 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.287.758 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.289.534 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1563 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.302.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 59, total running count: 1463 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.303.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.304.867 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1564 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.317.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 60, total running count: 1464 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.318.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.321.633 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1565 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.333.145 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 61, total running count: 1465 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.334.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.335.936 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1566 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.348.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 62, total running count: 1466 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.349.853 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.351.440 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1567 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.364.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 63, total running count: 1467 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.365.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.367.898 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1568 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.379.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 64, total running count: 1468 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.380.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.383.326 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1569 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.395.271 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 65, total running count: 1469 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.396.432 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.398.730 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1570 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.410.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 66, total running count: 1470 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.411.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.414.403 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1571 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.426.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 67, total running count: 1471 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.427.348 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.429.727 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1572 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.441.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 68, total running count: 1472 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.442.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.445.294 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1573 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.456.946 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 69, total running count: 1473 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.457.888 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.459.694 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1574 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.472.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 70, total running count: 1474 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.473.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.475.556 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1575 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.487.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 71, total running count: 1475 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.488.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.491.229 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1576 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.503.120 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 72, total running count: 1476 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.504.331 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.506.651 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1577 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.518.739 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 73, total running count: 1477 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.519.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.521.977 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1578 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.533.898 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 74, total running count: 1478 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.535.037 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.537.509 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1579 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.549.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 75, total running count: 1479 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:50.552.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.555.289 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1580 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.567.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 76, total running count: 1480 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.567.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.569.633 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1581 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.582.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 77, total running count: 1481 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.583.091 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.584.930 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1582 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.597.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 78, total running count: 1482 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.598.569 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.600.295 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1583 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.612.874 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 79, total running count: 1483 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.614.007 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.615.795 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1584 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.628.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 80, total running count: 1484 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:50.629.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.631.316 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1585 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.644.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 81, total running count: 1485 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.645.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.646.811 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1586 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.659.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 82, total running count: 1486 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.661.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.663.429 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1587 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.675.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 83, total running count: 1487 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.676.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.678.989 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1588 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.691.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 84, total running count: 1488 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.692.178 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.694.360 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1589 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.706.935 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 85, total running count: 1489 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.707.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.709.734 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1590 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.722.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 86, total running count: 1490 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.723.350 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.725.174 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1591 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.737.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 87, total running count: 1491 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.738.859 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.740.435 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1592 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.753.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 88, total running count: 1492 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.754.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.755.913 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1593 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.768.681 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 89, total running count: 1493 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.769.876 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.772.445 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1594 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.784.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 90, total running count: 1494 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.785.347 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.787.958 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1595 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.799.724 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 91, total running count: 1495 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.801.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.803.333 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1596 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.815.391 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 92, total running count: 1496 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.816.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.818.845 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1597 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.831.791 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 93, total running count: 1497 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.832.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.834.537 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1598 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.847.948 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 94, total running count: 1498 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.848.292 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.850.337 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1599 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.862.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 95, total running count: 1499 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.863.515 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.866.084 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1600 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:50.878.858 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 96, total running count: 1500 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.879.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.881.663 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1601 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.894.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 97, total running count: 1501 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.894.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.897.219 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1602 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.909.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 98, total running count: 1502 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.910.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.912.532 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1603 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.924.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 99, total running count: 1503 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.925.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.927.930 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1604 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.939.777 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 100, total running count: 1504 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.940.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.943.239 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1605 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:50.955.720 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 101, total running count: 1505 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.956.532 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.958.716 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1606 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.971.570 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 102, total running count: 1506 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.971.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.974.248 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1607 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:50.986.624 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 103, total running count: 1507 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:50.987.417 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:50.989.907 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1608 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.002.371 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 104, total running count: 1508 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.002.993 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.005.633 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1609 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.017.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 105, total running count: 1509 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.018.606 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.020.480 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1610 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.033.146 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 106, total running count: 1510 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.034.109 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.035.959 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1611 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.048.756 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 107, total running count: 1511 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.049.785 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.051.676 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1612 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.064.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 108, total running count: 1512 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.065.622 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.068.242 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1613 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.079.982 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 109, total running count: 1513 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.081.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.083.700 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1614 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.095.396 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 110, total running count: 1514 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.096.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.098.196 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1615 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.111.423 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 111, total running count: 1515 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.112.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.114.622 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1616 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.126.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 112, total running count: 1516 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:51.127.702 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.130.161 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1617 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.142.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 113, total running count: 1517 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.143.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.145.767 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1618 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.157.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 114, total running count: 1518 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.158.735 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.161.161 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1619 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.173.086 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 115, total running count: 1519 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.174.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.176.518 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1620 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.188.404 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 116, total running count: 1520 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.189.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.191.894 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1621 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.204.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 117, total running count: 1521 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.205.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.207.284 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1622 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.219.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 118, total running count: 1522 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.220.722 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.222.815 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1623 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.234.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 119, total running count: 1523 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.235.988 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.238.281 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1624 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.250.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 120, total running count: 1524 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.251.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.254.090 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1625 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.266.347 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 121, total running count: 1525 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.267.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.270.169 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1626 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.282.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 122, total running count: 1526 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.283.031 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.284.639 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1627 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.297.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 123, total running count: 1527 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.298.675 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.301.468 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1628 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.313.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 124, total running count: 1528 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.314.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.315.995 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1629 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.328.526 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 125, total running count: 1529 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.329.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.332.030 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1630 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.343.944 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 126, total running count: 1530 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.345.016 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.347.707 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1631 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.359.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 127, total running count: 1531 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.360.285 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.362.165 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1632 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.374.660 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 128, total running count: 1532 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.375.547 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.377.657 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1633 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.389.888 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 129, total running count: 1533 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.391.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.393.764 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1634 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.405.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 130, total running count: 1534 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.406.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.408.134 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1635 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.420.768 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 131, total running count: 1535 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.421.973 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.423.637 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1636 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.436.191 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 132, total running count: 1536 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.437.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.440.311 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1637 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.452.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 133, total running count: 1537 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.453.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.455.740 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1638 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.468.150 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 134, total running count: 1538 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.468.951 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.471.296 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1639 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:51.483.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 135, total running count: 1539 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.484.640 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.486.791 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1640 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.499.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 136, total running count: 1540 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.500.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.502.254 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1641 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.514.930 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 137, total running count: 1541 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.515.730 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.517.824 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1642 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.530.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 138, total running count: 1542 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.531.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.533.370 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1643 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.545.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 139, total running count: 1543 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.546.782 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.549.258 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1644 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.561.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 140, total running count: 1544 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.562.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.564.966 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1645 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.577.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 141, total running count: 1545 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.577.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.579.323 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1646 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.592.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 142, total running count: 1546 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.593.168 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.594.935 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1647 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.607.588 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 143, total running count: 1547 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.608.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.610.527 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1648 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.623.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 144, total running count: 1548 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.624.203 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.626.243 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1649 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.638.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 145, total running count: 1549 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.639.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.642.232 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1650 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.654.130 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 146, total running count: 1550 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.655.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.656.634 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1651 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.669.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 147, total running count: 1551 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.670.599 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.672.515 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1652 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.684.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 148, total running count: 1552 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.685.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.688.368 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1653 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.700.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 149, total running count: 1553 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.701.417 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.703.971 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1654 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.715.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 150, total running count: 1554 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.716.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.719.494 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1655 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.731.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 151, total running count: 1555 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.732.516 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.734.274 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1656 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.746.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 152, total running count: 1556 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.747.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.749.882 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1657 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.762.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 153, total running count: 1557 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.763.318 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.765.618 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1658 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.777.701 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 154, total running count: 1558 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.778.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.781.012 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1659 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.793.776 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 155, total running count: 1559 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.794.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.796.715 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1660 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.808.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 156, total running count: 1560 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.809.725 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.812.138 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1661 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.824.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 157, total running count: 1561 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.825.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.827.753 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1662 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.839.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 158, total running count: 1562 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.840.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.842.499 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1663 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.854.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 159, total running count: 1563 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.856.135 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.858.256 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1664 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.870.614 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 160, total running count: 1564 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.871.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.873.763 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1665 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.886.278 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 161, total running count: 1565 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.887.109 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.889.358 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1666 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.901.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 162, total running count: 1566 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.902.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.905.236 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1667 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.917.136 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 163, total running count: 1567 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.917.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.919.781 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1668 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.932.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 164, total running count: 1568 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.933.413 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.935.200 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1669 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.947.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 165, total running count: 1569 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.948.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.950.807 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1670 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.963.097 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 166, total running count: 1570 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.964.380 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.966.255 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1671 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:51.978.749 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 167, total running count: 1571 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.979.751 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.982.136 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1672 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:51.993.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 168, total running count: 1572 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:51.995.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:51.996.632 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1673 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.009.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 169, total running count: 1573 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.010.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.012.518 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1674 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.024.818 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 170, total running count: 1574 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.025.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.027.937 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1675 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.040.474 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 171, total running count: 1575 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.041.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.043.390 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1676 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.055.645 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 172, total running count: 1576 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.056.681 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.058.783 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1677 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.071.107 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 173, total running count: 1577 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.072.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.074.066 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1678 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.086.466 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 174, total running count: 1578 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.087.661 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.089.480 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1679 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.102.048 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 175, total running count: 1579 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.103.176 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.105.269 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1680 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.117.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 176, total running count: 1580 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.118.575 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.120.871 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1681 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.132.719 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 177, total running count: 1581 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.133.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.135.670 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1682 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.148.252 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 178, total running count: 1582 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.149.267 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.151.125 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1683 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.164.203 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 179, total running count: 1583 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.164.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.166.565 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1684 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.178.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 180, total running count: 1584 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.180.141 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.182.232 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1685 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.194.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 181, total running count: 1585 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.195.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.197.717 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1686 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.210.164 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 182, total running count: 1586 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.211.066 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.213.285 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1687 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.225.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 183, total running count: 1587 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.226.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.228.158 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1688 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.241.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 184, total running count: 1588 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.241.900 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.243.811 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1689 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:52.256.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 185, total running count: 1589 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.257.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.259.374 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1690 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.271.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 186, total running count: 1590 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.272.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.274.813 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1691 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.286.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 187, total running count: 1591 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.288.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.290.200 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1692 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.302.724 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 188, total running count: 1592 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.303.868 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.305.645 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1693 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.318.147 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 189, total running count: 1593 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.319.281 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.321.220 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1694 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.333.480 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 190, total running count: 1594 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.334.586 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.336.569 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1695 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.348.937 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 191, total running count: 1595 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.350.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.353.225 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1696 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.364.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 192, total running count: 1596 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.366.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.368.491 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1697 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.380.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 193, total running count: 1597 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.381.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.384.014 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1698 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.396.254 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 194, total running count: 1598 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.397.613 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.399.445 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1699 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.411.843 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 195, total running count: 1599 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.413.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.415.870 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1700 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.427.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 196, total running count: 1600 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.428.871 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.431.245 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1701 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.443.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 197, total running count: 1601 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.444.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.446.740 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1702 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.459.039 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 198, total running count: 1602 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.459.973 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.462.268 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1703 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.474.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 199, total running count: 1603 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.475.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.477.877 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1704 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.489.827 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 200, total running count: 1604 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.491.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.493.350 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1705 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.505.203 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 201, total running count: 1605 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.506.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.508.730 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1706 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.520.756 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 202, total running count: 1606 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.522.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.524.376 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1707 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.536.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 203, total running count: 1607 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.537.582 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.539.202 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1708 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.551.890 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 204, total running count: 1608 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.553.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.554.845 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1709 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.567.589 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 205, total running count: 1609 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.568.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.570.337 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1710 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.583.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 206, total running count: 1610 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.583.947 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.585.789 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1711 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.598.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 207, total running count: 1611 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.599.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.601.311 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1712 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.613.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 208, total running count: 1612 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.615.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.617.026 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1713 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.629.354 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 209, total running count: 1613 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.630.433 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.632.894 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1714 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.645.024 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 210, total running count: 1614 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.645.908 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.648.595 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1715 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.660.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 211, total running count: 1615 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:52.661.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.664.208 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1716 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.676.543 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 212, total running count: 1616 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.677.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.679.725 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1717 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.691.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 213, total running count: 1617 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.692.623 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.695.215 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1718 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.706.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 214, total running count: 1618 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.708.024 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.709.574 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1719 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.722.737 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 215, total running count: 1619 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.723.641 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.726.213 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1720 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.738.724 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 216, total running count: 1620 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.739.112 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.740.639 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1721 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.753.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 217, total running count: 1621 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.754.489 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.756.392 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1722 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.768.770 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 218, total running count: 1622 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.770.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.771.804 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1723 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.784.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 219, total running count: 1623 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.785.525 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.787.434 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1724 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.799.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 220, total running count: 1624 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.800.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.803.057 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1725 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.815.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 221, total running count: 1625 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.816.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.818.962 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1726 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.831.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 222, total running count: 1626 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.832.071 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.834.530 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1727 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.846.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 223, total running count: 1627 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.847.663 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.849.803 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1728 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.862.052 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 224, total running count: 1628 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.863.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.865.197 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1729 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.877.638 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 225, total running count: 1629 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.878.792 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.880.635 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1730 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.893.219 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 226, total running count: 1630 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.895.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.897.542 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1731 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.910.661 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 227, total running count: 1631 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.911.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.913.310 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1732 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:52.925.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 228, total running count: 1632 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.926.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.929.184 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1733 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.940.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 229, total running count: 1633 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.942.138 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.944.575 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1734 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.956.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 230, total running count: 1634 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.957.523 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.960.194 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1735 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.971.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 231, total running count: 1635 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.973.116 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.975.671 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1736 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:52.987.455 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 232, total running count: 1636 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:52.988.625 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:52.990.903 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1737 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.003.169 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 233, total running count: 1637 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.004.037 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.006.611 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1738 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.019.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 234, total running count: 1638 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.019.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.022.275 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1739 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.034.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 235, total running count: 1639 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.035.147 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.037.834 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1740 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.049.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 236, total running count: 1640 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.050.559 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.052.583 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1741 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.064.909 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 237, total running count: 1641 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.065.909 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.068.150 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1742 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.080.058 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 238, total running count: 1642 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.081.243 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.083.636 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1743 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.095.975 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 239, total running count: 1643 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.096.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.099.276 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1744 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.111.471 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 240, total running count: 1644 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.112.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.114.762 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1745 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.126.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 241, total running count: 1645 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.127.728 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.129.445 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1746 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.142.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 242, total running count: 1646 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.143.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.145.206 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1747 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.157.695 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 243, total running count: 1647 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.158.943 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.161.221 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1748 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.173.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 244, total running count: 1648 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.174.550 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.176.660 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1749 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.188.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 245, total running count: 1649 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.190.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.192.325 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1750 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.204.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 246, total running count: 1650 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.205.744 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.208.048 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1751 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.220.039 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 247, total running count: 1651 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.221.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.223.744 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1752 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.235.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 248, total running count: 1652 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.236.818 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.238.507 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1753 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.251.495 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 249, total running count: 1653 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.252.442 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.254.257 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1754 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.266.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 250, total running count: 1654 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.267.744 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.269.706 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1755 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.283.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 251, total running count: 1655 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.283.420 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.285.361 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1756 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.297.925 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 252, total running count: 1656 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.298.879 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.300.853 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1757 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.313.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 253, total running count: 1657 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.314.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.316.421 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1758 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:53.328.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 254, total running count: 1658 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:53.329.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.331.765 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1759 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.344.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 255, total running count: 1659 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.345.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.347.416 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1760 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.359.831 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 256, total running count: 1660 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.360.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.362.746 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1761 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.375.016 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 257, total running count: 1661 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.376.417 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.378.257 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1762 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.390.837 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 258, total running count: 1662 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.391.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.393.876 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1763 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.406.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 259, total running count: 1663 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.407.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.409.769 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1764 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.421.987 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 260, total running count: 1664 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.423.218 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.425.598 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1765 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.437.400 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 261, total running count: 1665 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.438.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.441.187 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1766 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.452.798 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 262, total running count: 1666 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.454.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.456.604 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1767 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.468.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 263, total running count: 1667 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.470.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.472.380 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1768 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:53.484.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 264, total running count: 1668 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:53.485.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.488.261 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1769 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.500.619 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 265, total running count: 1669 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.501.496 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.503.989 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1770 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.516.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 266, total running count: 1670 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.517.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.519.416 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1771 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.531.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 267, total running count: 1671 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.532.679 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.534.850 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1772 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.547.089 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 268, total running count: 1672 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.548.176 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.550.491 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1773 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.562.582 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 269, total running count: 1673 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.563.679 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.566.099 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1774 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.578.495 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 270, total running count: 1674 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.579.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.581.657 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1775 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.593.711 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 271, total running count: 1675 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.594.942 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.597.254 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1776 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.609.191 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 272, total running count: 1676 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.610.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.612.980 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1777 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.624.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 273, total running count: 1677 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.625.752 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.627.342 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1778 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.640.168 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 274, total running count: 1678 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.641.105 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.643.199 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1779 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.655.606 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 275, total running count: 1679 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.656.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.658.744 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1780 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.671.281 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 276, total running count: 1680 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.672.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.674.366 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1781 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.686.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 277, total running count: 1681 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:53.687.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.690.005 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1782 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.702.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 278, total running count: 1682 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.704.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.706.677 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1783 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.718.701 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 279, total running count: 1683 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.721.217 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.723.282 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1784 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.735.780 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 280, total running count: 1684 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.736.861 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.738.692 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1785 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.751.235 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 281, total running count: 1685 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.752.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.755.320 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1786 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.767.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 282, total running count: 1686 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.768.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.770.784 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1787 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.783.051 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 283, total running count: 1687 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.784.148 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.786.255 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1788 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.799.236 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 284, total running count: 1688 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.799.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.801.974 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1789 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.814.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 285, total running count: 1689 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.815.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.817.791 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1790 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.829.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 286, total running count: 1690 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.830.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.832.555 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1791 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:53.845.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 287, total running count: 1691 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.846.755 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.849.297 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1792 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.861.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 288, total running count: 1692 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.862.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.863.695 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1793 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.876.480 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 289, total running count: 1693 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.877.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.879.345 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1794 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.891.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 290, total running count: 1694 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.892.945 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.894.620 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1795 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.907.544 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 291, total running count: 1695 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.908.638 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.911.260 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1796 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.922.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 292, total running count: 1696 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.924.054 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.926.733 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1797 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.938.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 293, total running count: 1697 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.939.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.942.110 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1798 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.954.059 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 294, total running count: 1698 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.954.968 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.956.557 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1799 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:53.969.302 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 295, total running count: 1699 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.970.670 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.973.014 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1800 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:53.985.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 296, total running count: 1700 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:53.986.613 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:53.988.528 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1801 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.000.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 297, total running count: 1701 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:54.002.248 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.004.033 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1802 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.017.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 298, total running count: 1702 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.017.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.019.575 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1803 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.032.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 299, total running count: 1703 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.033.371 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.034.919 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1804 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.047.751 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 300, total running count: 1704 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.048.817 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.050.460 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1805 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.063.135 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 301, total running count: 1705 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.064.153 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.066.477 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1806 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.078.458 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 302, total running count: 1706 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.079.711 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.082.271 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1807 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:54.094.115 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 303, total running count: 1707 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:54.095.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.097.938 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1808 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.110.087 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 304, total running count: 1708 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.110.730 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.112.470 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1809 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.125.394 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 305, total running count: 1709 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.126.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.128.309 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1810 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.140.680 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 306, total running count: 1710 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.141.608 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.143.984 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1811 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.156.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 307, total running count: 1711 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.157.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.159.656 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1812 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.171.657 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 308, total running count: 1712 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.172.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.175.639 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1813 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.187.200 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 309, total running count: 1713 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.188.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.190.136 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1814 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.202.587 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 310, total running count: 1714 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.203.737 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.205.746 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1815 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.218.072 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 311, total running count: 1715 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.219.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.221.233 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1816 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.233.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 312, total running count: 1716 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.234.949 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.237.013 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1817 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.249.478 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 313, total running count: 1717 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.250.388 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.252.909 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1818 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.264.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 314, total running count: 1718 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.266.048 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.268.639 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1819 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.280.496 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 315, total running count: 1719 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.281.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.284.091 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1820 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.296.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 316, total running count: 1720 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.297.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.299.709 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1821 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.311.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 317, total running count: 1721 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.313.271 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.315.347 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1822 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.328.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 318, total running count: 1722 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.328.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.331.232 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1823 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.343.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 319, total running count: 1723 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.344.300 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.346.896 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1824 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.358.690 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 320, total running count: 1724 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.359.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.362.421 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1825 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:54.374.424 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 321, total running count: 1725 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:54.375.343 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.377.009 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1826 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.390.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 322, total running count: 1726 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.390.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.392.709 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1827 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.405.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 323, total running count: 1727 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.406.331 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.408.299 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1828 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.420.630 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 324, total running count: 1728 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.421.843 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.423.947 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1829 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.436.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 325, total running count: 1729 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.437.371 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.439.481 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1830 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.452.099 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 326, total running count: 1730 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.453.187 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.455.321 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1831 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.467.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 327, total running count: 1731 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.469.077 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.471.305 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1832 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.484.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 328, total running count: 1732 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.484.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.487.473 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1833 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.499.642 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 329, total running count: 1733 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.500.686 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.503.461 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1834 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.515.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 330, total running count: 1734 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.516.466 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.518.218 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1835 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.531.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 331, total running count: 1735 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.532.420 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.535.042 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1836 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:54.546.947 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 332, total running count: 1736 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.548.151 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.549.851 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1837 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.563.238 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 333, total running count: 1737 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.563.782 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.565.626 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1838 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.578.493 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 334, total running count: 1738 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.579.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.581.565 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1839 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.594.219 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 335, total running count: 1739 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.595.210 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.597.674 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1840 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.609.582 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 336, total running count: 1740 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.610.795 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.613.494 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1841 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.625.385 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 337, total running count: 1741 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.626.751 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.629.581 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1842 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.641.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 338, total running count: 1742 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.642.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.644.094 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1843 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.657.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 339, total running count: 1743 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.658.138 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.660.085 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1844 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.673.100 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 340, total running count: 1744 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.673.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.675.857 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1845 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.688.649 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 341, total running count: 1745 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.689.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.691.780 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1846 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.704.816 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 342, total running count: 1746 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.705.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.707.945 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1847 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.720.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 343, total running count: 1747 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.721.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.724.464 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1848 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.736.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 344, total running count: 1748 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.737.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.739.229 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1849 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.752.089 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 345, total running count: 1749 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.753.153 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.754.862 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1850 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.767.640 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 346, total running count: 1750 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.768.962 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.770.696 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1851 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.783.503 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 347, total running count: 1751 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.784.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.787.043 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1852 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.799.319 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 348, total running count: 1752 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.800.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.803.429 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1853 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.815.079 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 349, total running count: 1753 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.816.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.818.450 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1854 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.831.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 350, total running count: 1754 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.832.961 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.835.060 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1855 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.847.645 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 351, total running count: 1755 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.851.275 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.853.505 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1856 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.865.777 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 352, total running count: 1756 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.868.587 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.870.781 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1857 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.883.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 353, total running count: 1757 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.884.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.886.612 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1858 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.898.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 354, total running count: 1758 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.900.468 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.902.464 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1859 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.915.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 355, total running count: 1759 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.916.345 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.918.828 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1860 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.930.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 356, total running count: 1760 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.932.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.934.577 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1861 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.947.559 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 357, total running count: 1761 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.948.224 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.950.215 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1862 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.962.716 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 358, total running count: 1762 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.964.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.966.903 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1863 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:54.978.722 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 359, total running count: 1763 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.980.051 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.982.397 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1864 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:54.994.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 360, total running count: 1764 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:54.995.785 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:54.998.248 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1865 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.010.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 361, total running count: 1765 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.012.544 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.015.297 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1866 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.027.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 362, total running count: 1766 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.028.796 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.030.855 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1867 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.043.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 363, total running count: 1767 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.044.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.046.628 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1868 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.059.637 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 364, total running count: 1768 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.062.236 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.064.901 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1869 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.076.775 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 365, total running count: 1769 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.078.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.080.913 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1870 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:55.092.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 366, total running count: 1770 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:55.094.111 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.096.939 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1871 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.108.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 367, total running count: 1771 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.109.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.111.460 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1872 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.124.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 368, total running count: 1772 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.125.313 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.127.070 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1873 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.139.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 369, total running count: 1773 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.141.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.143.702 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1874 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.155.331 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 370, total running count: 1774 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.156.665 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.158.477 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1875 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.171.843 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 371, total running count: 1775 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.172.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.174.139 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1876 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.187.115 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 372, total running count: 1776 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.190.731 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.192.997 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1877 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.205.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 373, total running count: 1777 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.206.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.208.617 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1878 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.220.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 374, total running count: 1778 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.221.964 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.224.190 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1879 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.236.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 375, total running count: 1779 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.237.572 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.239.884 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1880 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.251.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 376, total running count: 1780 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.253.165 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.255.702 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1881 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.267.419 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 377, total running count: 1781 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.268.658 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.270.250 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1882 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.283.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 378, total running count: 1782 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.284.122 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.285.760 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1883 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.299.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 379, total running count: 1783 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.299.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.301.821 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1884 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:55.314.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 380, total running count: 1784 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.315.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.317.724 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1885 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.330.065 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 381, total running count: 1785 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.331.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.333.241 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1886 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.345.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 382, total running count: 1786 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.347.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.349.791 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1887 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.361.522 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 383, total running count: 1787 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.363.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.366.580 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1888 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.378.147 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 384, total running count: 1788 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.379.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.382.277 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1889 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.394.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 385, total running count: 1789 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.395.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.398.138 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1890 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.409.759 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 386, total running count: 1790 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.412.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.415.198 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1891 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.427.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 387, total running count: 1791 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.429.213 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.431.151 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1892 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.443.533 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 388, total running count: 1792 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.445.020 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.446.758 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1893 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.459.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 389, total running count: 1793 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.460.997 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.463.507 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1894 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.476.218 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 390, total running count: 1794 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.476.791 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.479.061 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1895 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.491.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 391, total running count: 1795 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.492.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.494.905 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1896 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.507.382 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 392, total running count: 1796 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.508.224 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.510.710 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1897 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.522.903 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 393, total running count: 1797 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.524.572 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.526.371 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1898 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:55.539.011 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 394, total running count: 1798 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:55.541.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.543.229 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1899 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.556.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 395, total running count: 1799 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.557.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.560.375 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1900 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.572.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 396, total running count: 1800 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.573.508 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.576.079 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1901 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.587.860 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 397, total running count: 1801 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.589.488 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.591.215 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1902 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.604.031 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 398, total running count: 1802 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.605.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.608.089 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1903 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.620.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 399, total running count: 1803 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.621.541 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.623.493 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1904 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.635.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 400, total running count: 1804 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.637.264 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.638.814 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1905 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.652.111 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 401, total running count: 1805 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.654.358 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.656.642 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1906 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.668.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 402, total running count: 1806 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.671.679 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.673.662 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1907 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.686.031 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 403, total running count: 1807 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.687.863 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.689.799 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1908 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.702.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 404, total running count: 1808 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.703.822 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.705.461 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1909 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.718.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 405, total running count: 1809 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.719.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.722.359 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1910 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.734.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 406, total running count: 1810 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.735.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.738.331 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1911 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.750.748 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 407, total running count: 1811 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.751.969 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.753.905 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1912 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.766.427 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 408, total running count: 1812 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.768.182 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.770.673 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1913 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.782.601 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 409, total running count: 1813 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.783.909 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.786.450 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1914 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.798.689 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 410, total running count: 1814 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.799.813 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.802.083 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1915 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.814.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 411, total running count: 1815 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.815.777 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.818.415 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1916 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.830.114 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 412, total running count: 1816 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.831.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.833.839 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1917 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:55.846.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 413, total running count: 1817 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.847.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.849.377 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1918 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.862.165 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 414, total running count: 1818 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.863.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.865.261 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1919 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.877.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 415, total running count: 1819 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.878.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.880.819 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1920 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.893.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 416, total running count: 1820 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.895.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.897.673 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1921 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:55.910.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 417, total running count: 1821 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.910.740 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.913.305 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1922 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.925.453 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 418, total running count: 1822 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:55.926.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.929.009 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1923 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.941.131 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 419, total running count: 1823 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.942.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.944.888 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1924 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.956.933 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 420, total running count: 1824 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.958.292 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.960.653 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1925 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.972.481 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 421, total running count: 1825 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:55.974.086 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.976.346 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1926 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.988.674 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 422, total running count: 1826 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:55.989.982 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:55.991.685 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1927 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.004.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 423, total running count: 1827 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.005.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.007.490 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1928 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.019.949 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 424, total running count: 1828 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.021.424 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.023.198 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1929 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.035.735 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 425, total running count: 1829 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.037.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.038.772 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1930 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.051.813 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 426, total running count: 1830 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.053.303 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.055.759 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1931 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.067.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 427, total running count: 1831 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.069.408 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.071.286 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1932 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.084.058 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 428, total running count: 1832 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.085.437 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.087.045 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1933 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.099.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 429, total running count: 1833 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.101.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.102.765 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1934 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.115.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 430, total running count: 1834 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.116.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.118.547 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1935 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.131.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 431, total running count: 1835 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.132.498 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.134.326 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1936 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.146.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 432, total running count: 1836 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.147.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.149.894 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1937 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.162.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 433, total running count: 1837 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.163.580 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.165.232 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1938 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.178.570 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 434, total running count: 1838 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.179.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.181.768 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1939 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.193.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 435, total running count: 1839 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.195.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.197.489 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1940 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.209.503 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 436, total running count: 1840 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.210.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.213.154 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1941 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.225.295 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 437, total running count: 1841 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.226.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.228.713 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1942 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.241.313 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 438, total running count: 1842 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.242.668 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.244.484 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1943 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.256.887 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 439, total running count: 1843 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.258.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.260.608 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1944 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.272.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 440, total running count: 1844 [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:56.273.804 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.276.091 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1945 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.288.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 441, total running count: 1845 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.289.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.291.706 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1946 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.303.856 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 442, total running count: 1846 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.305.024 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.307.416 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1947 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.319.389 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 443, total running count: 1847 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.320.853 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.323.273 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1948 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.335.387 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 444, total running count: 1848 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.336.665 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.338.857 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1949 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.350.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 445, total running count: 1849 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.352.678 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.354.455 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1950 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.367.533 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 446, total running count: 1850 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.368.535 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.370.541 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1951 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.382.936 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 447, total running count: 1851 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.384.219 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.386.366 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1952 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.398.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 448, total running count: 1852 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.399.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.401.891 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1953 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.414.396 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 449, total running count: 1853 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.415.850 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.417.647 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1954 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.430.399 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 450, total running count: 1854 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.431.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.433.811 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1955 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.445.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 451, total running count: 1855 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.447.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.449.549 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1956 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.461.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 452, total running count: 1856 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.462.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.465.240 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1957 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.477.202 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 453, total running count: 1857 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.478.381 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.480.699 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1958 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.493.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 454, total running count: 1858 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.494.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.496.090 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1959 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.508.660 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 455, total running count: 1859 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.509.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.511.900 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1960 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.524.131 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 456, total running count: 1860 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.525.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.527.795 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1961 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.539.658 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 457, total running count: 1861 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.541.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.543.319 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1962 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.555.558 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 458, total running count: 1862 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.556.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.558.753 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1963 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.571.949 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 459, total running count: 1863 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.572.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.574.232 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1964 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.586.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 460, total running count: 1864 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.588.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.589.805 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1965 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.602.714 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 461, total running count: 1865 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.603.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.605.775 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1966 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.617.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 462, total running count: 1866 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.619.300 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.621.629 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1967 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:56.633.765 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 463, total running count: 1867 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.634.933 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.637.281 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1968 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.649.425 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 464, total running count: 1868 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.650.485 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.653.051 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1969 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.665.142 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 465, total running count: 1869 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.665.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.667.915 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1970 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.680.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 466, total running count: 1870 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.681.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.683.552 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1971 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.696.455 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 467, total running count: 1871 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.696.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.699.206 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1972 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:56.712.177 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 468, total running count: 1872 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:40:56.712.638 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_1 execution count: 4, execution time: 7324.2 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:40:56.712.813 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_1 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:40:56.712.910 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty epoch: 4 step: 468, loss is 0.6061489582061768 [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:40:56.713.740 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.h:76] ContinueSend] continue send at the beginning of the epoch [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:40:56.714.601 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:40:56.714.658 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:40:56.714.698 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_1 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.715.028 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.717.041 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1973 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.729.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 1, total running count: 1873 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.731.678 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.733.782 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1974 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.746.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 2, total running count: 1874 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.747.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.749.493 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1975 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.762.174 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 3, total running count: 1875 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.763.111 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.765.578 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1976 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.777.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 4, total running count: 1876 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.778.580 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.780.451 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1977 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.792.933 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 5, total running count: 1877 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.794.129 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.796.123 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1978 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.808.970 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 6, total running count: 1878 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.809.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.811.693 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1979 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.824.254 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 7, total running count: 1879 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.825.503 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.827.451 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1980 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe43fff0f0,python):2024-01-10-11:40:56.840.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 8, total running count: 1880 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.841.105 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.842.939 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1981 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.858.483 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 9, total running count: 1881 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.858.839 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.861.384 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1982 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.874.238 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 10, total running count: 1882 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.874.582 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.876.859 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1983 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.889.455 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 11, total running count: 1883 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.889.808 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.892.353 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1984 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.904.466 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 12, total running count: 1884 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.904.796 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.906.637 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1985 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.919.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 13, total running count: 1885 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.919.630 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.922.123 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1986 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.934.251 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 14, total running count: 1886 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.934.627 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.936.503 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1987 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.949.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 15, total running count: 1887 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.949.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.951.893 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1988 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.964.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 16, total running count: 1888 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.965.444 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.967.361 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1989 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.980.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 17, total running count: 1889 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:56.980.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.983.128 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1990 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:56.995.574 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 18, total running count: 1890 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:56.996.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:56.998.934 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1991 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.011.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 19, total running count: 1891 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.012.177 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.014.811 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1992 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.026.570 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 20, total running count: 1892 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.027.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.029.377 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1993 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.042.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 21, total running count: 1893 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.042.909 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.044.812 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1994 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.057.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 22, total running count: 1894 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.058.569 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.060.317 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1995 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.073.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 23, total running count: 1895 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.074.304 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.076.223 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1996 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.089.146 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 24, total running count: 1896 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.090.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.091.932 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1997 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.105.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 25, total running count: 1897 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.105.785 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.107.472 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1998 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.120.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 26, total running count: 1898 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.121.375 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.124.154 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1999 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.136.096 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 27, total running count: 1899 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.136.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.138.957 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2000 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.151.432 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 28, total running count: 1900 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.152.728 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.154.892 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2001 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.167.231 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 29, total running count: 1901 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.168.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.170.277 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2002 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.183.988 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 30, total running count: 1902 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.184.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.186.853 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2003 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.198.700 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 31, total running count: 1903 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.199.134 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.201.021 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2004 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.214.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 32, total running count: 1904 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.214.623 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.216.449 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2005 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.229.812 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 33, total running count: 1905 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.230.214 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.231.944 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2006 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.246.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 34, total running count: 1906 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.246.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.248.437 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2007 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.261.508 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 35, total running count: 1907 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.261.928 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.263.882 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2008 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.276.619 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 36, total running count: 1908 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.277.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.279.409 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2009 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.292.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 37, total running count: 1909 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.292.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.294.981 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2010 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.307.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 38, total running count: 1910 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.308.755 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.311.418 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2011 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.323.419 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 39, total running count: 1911 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.324.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.325.997 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2012 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.338.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 40, total running count: 1912 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.339.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.341.853 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2013 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.354.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 41, total running count: 1913 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.355.943 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.357.781 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2014 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.371.080 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 42, total running count: 1914 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.372.020 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.374.279 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2015 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.386.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 43, total running count: 1915 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.387.709 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.389.585 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2016 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.402.262 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 44, total running count: 1916 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.403.244 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.405.151 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2017 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.417.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 45, total running count: 1917 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.418.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.420.484 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2018 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.432.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 46, total running count: 1918 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.434.108 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.435.962 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2019 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.448.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 47, total running count: 1919 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.449.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.452.293 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2020 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.463.899 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 48, total running count: 1920 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.465.154 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.466.665 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2021 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.479.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 49, total running count: 1921 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.480.879 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.483.287 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2022 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.495.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 50, total running count: 1922 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.496.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.498.981 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2023 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.511.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 51, total running count: 1923 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.512.317 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.514.576 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2024 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.526.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 52, total running count: 1924 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.527.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.530.610 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2025 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:57.542.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 53, total running count: 1925 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.543.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.545.015 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2026 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.557.769 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 54, total running count: 1926 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.558.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.560.922 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2027 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.573.300 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 55, total running count: 1927 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.574.511 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.576.512 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2028 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.588.734 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 56, total running count: 1928 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.590.265 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.592.253 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2029 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.604.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 57, total running count: 1929 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:57.605.840 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.607.677 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2030 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.620.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 58, total running count: 1930 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.621.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.623.301 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2031 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.635.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 59, total running count: 1931 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.637.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.639.162 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2032 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.651.465 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 60, total running count: 1932 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:57.652.755 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.655.002 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2033 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.666.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 61, total running count: 1933 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:57.668.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.670.705 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2034 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.682.425 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 62, total running count: 1934 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:57.683.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.685.367 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2035 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.697.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 63, total running count: 1935 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:57.699.346 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.701.238 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2036 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.713.535 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 64, total running count: 1936 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.714.750 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.716.653 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2037 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.728.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 65, total running count: 1937 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.730.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.732.157 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2038 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.744.663 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 66, total running count: 1938 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.746.479 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.748.560 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2039 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.760.526 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 67, total running count: 1939 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:57.762.225 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.763.999 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2040 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.776.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 68, total running count: 1940 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.777.654 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.779.552 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2041 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.791.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 69, total running count: 1941 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.793.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.794.878 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2042 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:57.807.399 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 70, total running count: 1942 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.808.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.810.391 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2043 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:57.822.947 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 71, total running count: 1943 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:57.824.443 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.827.073 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2044 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.838.651 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 72, total running count: 1944 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.840.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.842.618 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2045 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.854.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 73, total running count: 1945 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:57.855.388 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.857.283 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2046 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:57.869.742 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 74, total running count: 1946 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:57.871.008 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.873.229 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2047 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.888.813 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 75, total running count: 1947 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.889.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.891.280 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2048 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:57.903.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 76, total running count: 1948 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.903.793 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.905.902 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2049 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.918.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 77, total running count: 1949 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.918.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.920.606 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2050 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.932.732 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 78, total running count: 1950 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.934.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.936.054 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2051 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.948.620 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 79, total running count: 1951 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.950.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.952.539 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2052 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:57.964.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 80, total running count: 1952 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.965.387 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.967.883 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2053 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.979.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 81, total running count: 1953 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:57.980.827 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.983.270 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2054 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.995.331 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 82, total running count: 1954 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:57.996.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:57.998.801 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2055 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.010.453 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 83, total running count: 1955 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.012.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.014.269 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2056 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.026.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 84, total running count: 1956 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.027.560 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.029.751 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2057 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.041.613 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 85, total running count: 1957 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.042.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.045.401 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2058 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.057.243 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 86, total running count: 1958 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.058.641 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.061.256 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2059 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.072.809 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 87, total running count: 1959 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.074.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.075.784 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2060 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.088.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 88, total running count: 1960 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.089.642 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.091.249 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2061 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.103.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 89, total running count: 1961 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.105.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.106.708 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2062 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.119.275 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 90, total running count: 1962 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.120.687 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.122.282 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2063 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.135.572 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 91, total running count: 1963 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.136.186 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.137.817 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2064 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.150.811 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 92, total running count: 1964 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.151.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.153.588 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2065 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.165.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 93, total running count: 1965 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.166.970 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.169.251 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2066 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.181.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 94, total running count: 1966 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.182.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.184.914 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2067 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.196.771 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 95, total running count: 1967 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.198.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.200.427 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2068 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.212.517 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 96, total running count: 1968 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.213.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.215.856 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2069 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.227.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 97, total running count: 1969 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.229.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.231.356 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2070 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.243.248 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 98, total running count: 1970 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.244.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.246.912 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2071 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.258.817 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 99, total running count: 1971 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.260.096 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.262.498 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2072 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.274.275 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 100, total running count: 1972 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.275.535 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.277.265 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2073 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.289.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 101, total running count: 1973 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.291.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.292.957 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2074 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.305.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 102, total running count: 1974 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.306.971 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.308.503 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2075 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.321.099 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 103, total running count: 1975 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.322.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.325.009 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2076 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.336.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 104, total running count: 1976 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.338.018 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.340.432 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2077 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.352.079 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 105, total running count: 1977 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.353.434 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.355.931 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2078 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.367.629 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 106, total running count: 1978 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.368.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.371.590 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2079 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.385.091 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 107, total running count: 1979 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.385.474 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.387.271 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2080 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.400.304 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 108, total running count: 1980 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.400.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.403.046 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2081 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.414.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 109, total running count: 1981 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.415.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.417.572 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2082 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.429.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 110, total running count: 1982 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.431.186 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.433.031 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2083 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.445.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 111, total running count: 1983 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.446.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.448.356 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2084 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.460.657 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 112, total running count: 1984 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.462.106 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.464.105 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2085 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.476.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 113, total running count: 1985 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.477.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.479.627 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2086 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.491.854 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 114, total running count: 1986 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.493.304 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.495.358 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2087 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.507.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 115, total running count: 1987 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.508.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.510.750 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2088 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.522.742 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 116, total running count: 1988 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.523.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.526.471 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2089 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.538.142 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 117, total running count: 1989 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.539.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.541.958 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2090 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.553.616 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 118, total running count: 1990 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.554.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.556.467 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2091 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.568.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 119, total running count: 1991 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.570.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.572.273 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2092 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.584.511 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 120, total running count: 1992 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.585.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.587.990 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2093 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.599.660 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 121, total running count: 1993 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.601.235 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.603.297 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2094 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.615.544 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 122, total running count: 1994 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.617.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.618.872 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2095 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.631.695 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 123, total running count: 1995 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.632.389 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.634.504 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2096 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.646.687 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 124, total running count: 1996 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.648.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.650.467 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2097 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.662.374 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 125, total running count: 1997 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.663.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.666.167 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2098 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.678.375 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 126, total running count: 1998 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.679.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.681.939 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2099 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.693.516 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 127, total running count: 1999 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.694.975 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.697.219 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2100 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.709.222 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 128, total running count: 2000 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.710.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.712.572 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2101 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.725.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 129, total running count: 2001 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.725.973 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.727.964 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2102 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.742.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 130, total running count: 2002 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.742.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.744.768 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2103 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.756.945 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 131, total running count: 2003 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.757.271 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.759.216 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2104 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.771.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 132, total running count: 2004 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.772.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.774.994 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2105 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.786.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 133, total running count: 2005 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.787.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.789.415 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2106 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.802.235 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 134, total running count: 2006 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.803.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.805.992 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2107 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.817.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 135, total running count: 2007 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.818.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.821.516 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2108 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.833.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 136, total running count: 2008 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.834.043 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.835.729 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2109 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.848.372 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 137, total running count: 2009 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.849.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.851.144 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2110 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.863.751 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 138, total running count: 2010 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.864.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.866.728 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2111 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.879.199 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 139, total running count: 2011 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:58.880.181 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.882.254 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2112 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.894.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 140, total running count: 2012 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.895.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.897.586 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2113 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.910.015 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 141, total running count: 2013 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.911.526 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.914.197 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2114 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.925.768 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 142, total running count: 2014 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.927.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.929.525 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2115 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.941.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 143, total running count: 2015 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.942.495 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.945.062 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2116 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.956.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 144, total running count: 2016 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:58.957.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.960.496 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2117 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.972.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 145, total running count: 2017 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.973.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.975.955 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2118 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:58.987.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 146, total running count: 2018 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:58.988.948 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:58.991.653 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2119 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.003.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 147, total running count: 2019 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.004.382 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.006.443 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2120 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.018.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 148, total running count: 2020 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.019.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.022.158 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2121 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.033.971 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 149, total running count: 2021 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.035.252 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.037.617 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2122 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.049.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 150, total running count: 2022 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.050.885 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.053.174 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2123 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.065.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 151, total running count: 2023 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.066.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.068.531 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2124 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.080.698 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 152, total running count: 2024 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.082.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.083.857 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2125 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.096.437 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 153, total running count: 2025 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.097.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.099.326 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2126 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.112.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 154, total running count: 2026 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.113.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.114.754 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2127 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.127.146 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 155, total running count: 2027 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.128.388 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.130.117 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2128 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.142.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 156, total running count: 2028 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.143.864 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.145.746 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2129 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.158.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 157, total running count: 2029 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.159.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.161.325 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2130 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.173.541 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 158, total running count: 2030 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.174.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.176.953 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2131 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.189.030 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 159, total running count: 2031 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.190.356 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.192.863 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2132 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.204.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 160, total running count: 2032 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.205.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.208.300 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2133 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.219.997 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 161, total running count: 2033 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.221.301 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.223.099 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2134 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.235.488 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 162, total running count: 2034 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.236.930 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.238.830 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2135 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.250.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 163, total running count: 2035 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.252.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.254.772 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2136 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.266.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 164, total running count: 2036 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.268.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.270.138 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2137 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.282.134 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 165, total running count: 2037 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.283.308 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.285.739 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2138 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.297.551 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 166, total running count: 2038 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.298.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.301.160 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2139 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.313.028 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 167, total running count: 2039 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.314.514 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.316.594 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2140 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.328.637 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 168, total running count: 2040 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.330.001 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.332.008 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2141 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.344.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 169, total running count: 2041 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.345.554 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.347.473 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2142 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.359.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 170, total running count: 2042 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.360.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.362.892 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2143 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.375.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 171, total running count: 2043 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.376.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.378.372 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2144 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.390.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 172, total running count: 2044 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.391.898 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.393.628 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2145 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.406.181 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 173, total running count: 2045 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.407.434 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.409.016 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2146 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.421.670 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 174, total running count: 2046 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.423.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.425.600 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2147 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.437.127 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 175, total running count: 2047 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.438.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.441.280 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2148 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.452.704 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 176, total running count: 2048 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.453.908 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.455.627 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2149 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.468.079 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 177, total running count: 2049 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.469.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.471.110 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2150 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.483.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 178, total running count: 2050 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.484.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.486.859 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2151 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.499.092 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 179, total running count: 2051 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.500.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.502.324 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2152 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.514.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 180, total running count: 2052 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.515.885 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.517.911 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2153 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.529.970 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 181, total running count: 2053 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.531.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.533.356 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2154 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.545.740 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 182, total running count: 2054 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.547.329 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.549.325 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2155 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.561.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 183, total running count: 2055 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.563.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.565.008 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2156 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.577.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 184, total running count: 2056 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.578.971 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.580.496 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2157 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.593.286 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 185, total running count: 2057 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.594.809 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.597.161 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2158 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.608.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 186, total running count: 2058 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.610.303 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.612.467 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2159 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.624.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 187, total running count: 2059 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.625.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.627.942 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2160 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.639.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 188, total running count: 2060 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.641.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.643.398 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2161 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.655.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 189, total running count: 2061 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.657.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.658.719 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2162 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.671.147 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 190, total running count: 2062 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.672.889 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.675.243 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2163 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.687.169 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 191, total running count: 2063 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.688.440 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.690.742 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2164 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.702.837 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 192, total running count: 2064 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.703.989 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.706.140 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2165 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.718.290 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 193, total running count: 2065 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.719.756 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.721.723 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2166 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.733.997 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 194, total running count: 2066 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.735.349 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.737.382 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2167 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.749.432 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 195, total running count: 2067 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.750.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.753.242 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2168 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.764.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 196, total running count: 2068 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.766.265 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.768.758 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2169 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.780.558 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 197, total running count: 2069 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.781.653 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.784.222 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2170 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.795.791 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 198, total running count: 2070 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.797.275 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.799.797 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2171 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.811.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 199, total running count: 2071 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.812.750 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.815.314 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2172 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.827.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 200, total running count: 2072 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.828.250 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.830.650 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2173 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.842.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 201, total running count: 2073 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.843.696 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.846.553 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2174 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.857.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 202, total running count: 2074 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.859.468 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.861.274 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2175 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.873.560 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 203, total running count: 2075 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.874.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.876.938 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2176 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.889.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 204, total running count: 2076 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.890.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.892.430 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2177 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.904.602 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 205, total running count: 2077 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.905.770 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.907.861 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2178 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.919.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 206, total running count: 2078 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.921.246 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.923.428 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2179 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.935.398 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 207, total running count: 2079 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.936.731 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.938.751 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2180 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:40:59.950.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 208, total running count: 2080 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.952.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.954.217 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2181 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.966.362 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 209, total running count: 2081 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.967.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.969.769 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2182 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:40:59.981.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 210, total running count: 2082 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.983.382 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:40:59.985.188 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2183 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:40:59.997.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 211, total running count: 2083 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:40:59.999.078 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.001.886 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2184 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.013.207 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 212, total running count: 2084 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.014.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.017.315 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2185 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.028.840 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 213, total running count: 2085 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.030.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.031.692 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2186 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.044.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 214, total running count: 2086 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.045.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.047.626 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2187 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.060.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 215, total running count: 2087 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.061.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.063.373 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2188 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.075.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 216, total running count: 2088 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.077.459 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.079.992 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2189 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.091.627 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 217, total running count: 2089 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.093.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.095.576 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2190 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.107.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 218, total running count: 2090 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.108.764 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.110.932 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2191 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.123.001 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 219, total running count: 2091 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.124.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.126.310 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2192 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.138.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 220, total running count: 2092 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.139.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.141.742 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2193 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.154.202 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 221, total running count: 2093 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.155.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.158.448 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2194 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.170.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 222, total running count: 2094 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.171.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.173.627 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2195 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.185.633 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 223, total running count: 2095 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.187.225 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.189.146 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2196 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.201.260 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 224, total running count: 2096 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.202.874 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.204.516 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2197 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.217.184 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 225, total running count: 2097 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.218.493 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.221.206 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2198 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.232.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 226, total running count: 2098 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.233.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.236.455 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2199 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.247.955 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 227, total running count: 2099 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.249.544 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.251.794 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2200 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.263.751 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 228, total running count: 2100 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.265.370 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.267.402 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2201 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.279.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 229, total running count: 2101 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.281.127 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.282.935 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2202 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.295.105 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 230, total running count: 2102 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.296.776 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.298.394 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2203 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.310.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 231, total running count: 2103 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.312.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.314.776 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2204 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.326.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 232, total running count: 2104 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.328.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.330.219 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2205 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.342.285 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 233, total running count: 2105 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.343.663 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.345.834 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2206 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.357.988 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 234, total running count: 2106 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.359.447 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.361.507 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2207 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.373.580 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 235, total running count: 2107 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.375.163 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.377.392 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2208 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.389.200 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 236, total running count: 2108 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.390.942 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.393.233 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2209 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.405.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 237, total running count: 2109 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.406.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.408.899 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2210 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.420.785 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 238, total running count: 2110 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.422.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.424.792 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2211 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.436.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 239, total running count: 2111 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.437.680 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.439.397 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2212 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.451.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 240, total running count: 2112 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.453.281 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.454.936 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2213 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.467.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 241, total running count: 2113 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.468.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.470.428 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2214 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.482.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 242, total running count: 2114 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.484.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.485.971 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2215 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.498.623 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 243, total running count: 2115 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.499.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.501.651 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2216 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.513.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 244, total running count: 2116 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.515.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.517.196 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2217 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.529.572 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 245, total running count: 2117 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.531.032 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.532.581 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2218 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.545.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 246, total running count: 2118 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.546.575 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.549.254 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2219 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.560.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 247, total running count: 2119 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.562.114 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.563.680 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2220 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.576.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 248, total running count: 2120 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.577.592 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.579.231 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2221 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.591.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 249, total running count: 2121 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.593.156 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.594.900 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2222 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.607.352 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 250, total running count: 2122 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.608.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.610.668 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2223 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.622.647 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 251, total running count: 2123 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.623.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.626.509 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2224 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.637.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 252, total running count: 2124 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.639.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.640.983 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2225 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.653.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 253, total running count: 2125 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.655.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.657.582 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2226 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.669.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 254, total running count: 2126 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.670.379 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.673.007 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2227 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.684.493 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 255, total running count: 2127 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.686.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.688.500 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2228 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.700.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 256, total running count: 2128 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.701.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.703.997 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2229 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.715.768 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 257, total running count: 2129 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.717.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.719.305 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2230 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.731.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 258, total running count: 2130 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.733.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.736.001 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2231 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.747.568 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 259, total running count: 2131 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.748.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.751.411 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2232 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.762.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 260, total running count: 2132 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.764.287 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.766.754 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2233 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.778.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 261, total running count: 2133 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.780.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.782.278 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2234 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.794.503 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 262, total running count: 2134 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.795.696 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.797.973 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2235 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.809.758 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 263, total running count: 2135 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.811.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.813.711 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2236 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.825.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 264, total running count: 2136 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.826.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.829.534 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2237 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.841.169 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 265, total running count: 2137 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.842.627 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.845.244 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2238 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.856.711 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 266, total running count: 2138 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.858.121 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.860.630 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2239 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.872.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 267, total running count: 2139 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.873.640 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.876.238 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2240 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.887.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 268, total running count: 2140 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.889.236 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.891.649 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2241 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.903.347 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 269, total running count: 2141 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.905.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.907.611 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2242 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.919.232 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 270, total running count: 2142 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.920.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.923.030 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2243 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.934.912 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 271, total running count: 2143 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:00.936.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.938.669 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2244 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.950.246 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 272, total running count: 2144 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.951.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.953.427 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2245 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:00.965.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 273, total running count: 2145 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.967.265 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.969.141 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2246 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.981.398 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 274, total running count: 2146 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.982.859 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:00.984.490 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2247 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:00.996.968 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 275, total running count: 2147 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:00.998.613 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.000.950 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2248 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.012.912 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 276, total running count: 2148 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.014.270 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.016.373 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2249 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.028.622 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 277, total running count: 2149 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.029.744 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.031.799 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2250 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.043.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 278, total running count: 2150 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.045.184 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.047.308 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2251 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.059.329 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 279, total running count: 2151 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.060.740 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.062.901 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2252 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.074.778 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 280, total running count: 2152 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.076.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.078.403 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2253 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.090.559 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 281, total running count: 2153 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.091.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.093.662 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2254 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.105.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 282, total running count: 2154 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.107.286 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.109.132 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2255 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.121.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 283, total running count: 2155 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.122.883 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.124.824 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2256 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.136.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 284, total running count: 2156 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.138.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.140.583 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2257 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.152.649 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 285, total running count: 2157 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.154.202 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.156.378 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2258 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.168.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 286, total running count: 2158 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.169.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.171.947 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2259 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.183.766 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 287, total running count: 2159 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.185.277 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.187.715 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2260 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.199.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 288, total running count: 2160 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.200.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.203.260 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2261 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.215.151 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 289, total running count: 2161 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.216.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.218.923 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2262 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.230.441 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 290, total running count: 2162 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.231.811 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.233.382 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2263 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.245.792 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 291, total running count: 2163 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.247.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.248.935 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2264 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.261.510 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 292, total running count: 2164 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.262.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.264.597 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2265 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.277.030 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 293, total running count: 2165 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.278.353 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.280.480 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2266 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.292.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 294, total running count: 2166 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.293.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.296.238 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2267 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.307.858 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 295, total running count: 2167 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.309.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.311.704 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2268 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.323.556 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 296, total running count: 2168 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.324.898 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.327.243 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2269 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.339.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 297, total running count: 2169 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.340.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.342.679 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2270 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.354.818 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 298, total running count: 2170 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.355.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.358.230 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2271 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.370.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 299, total running count: 2171 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.371.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.373.755 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2272 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.385.594 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 300, total running count: 2172 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.386.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.389.242 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2273 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.400.966 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 301, total running count: 2173 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.402.486 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.404.814 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2274 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.416.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 302, total running count: 2174 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.418.265 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.420.419 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2275 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.432.374 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 303, total running count: 2175 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.433.938 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.436.284 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2276 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.448.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 304, total running count: 2176 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.449.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.451.673 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2277 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.463.541 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 305, total running count: 2177 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.464.817 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.467.129 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2278 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.479.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 306, total running count: 2178 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.480.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.482.640 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2279 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.494.130 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 307, total running count: 2179 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.495.545 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.498.056 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2280 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.509.658 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 308, total running count: 2180 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.510.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.512.433 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2281 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.525.079 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 309, total running count: 2181 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.526.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.528.927 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2282 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.540.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 310, total running count: 2182 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.541.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.544.416 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2283 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.556.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 311, total running count: 2183 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.557.356 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.558.916 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2284 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.571.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 312, total running count: 2184 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.572.644 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.574.694 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2285 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.586.960 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 313, total running count: 2185 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.588.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.590.552 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2286 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.602.644 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 314, total running count: 2186 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.603.625 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.606.150 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2287 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.617.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 315, total running count: 2187 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.619.011 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.621.578 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2288 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.633.177 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 316, total running count: 2188 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.634.523 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.636.882 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2289 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.648.691 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 317, total running count: 2189 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.649.848 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.652.511 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2290 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.663.969 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 318, total running count: 2190 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.665.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.667.206 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2291 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.679.319 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 319, total running count: 2191 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.680.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.683.088 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2292 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.694.700 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 320, total running count: 2192 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.696.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.697.551 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2293 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.710.106 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 321, total running count: 2193 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.711.696 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.714.200 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2294 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.725.874 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 322, total running count: 2194 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.727.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.729.567 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2295 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.741.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 323, total running count: 2195 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.742.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.744.963 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2296 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.756.527 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 324, total running count: 2196 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.757.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.760.334 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2297 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.771.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 325, total running count: 2197 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.773.437 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.775.750 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2298 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.787.398 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 326, total running count: 2198 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.788.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.791.200 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2299 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.803.116 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 327, total running count: 2199 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.804.539 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.806.648 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2300 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.818.583 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 328, total running count: 2200 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.819.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.822.118 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2301 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.834.052 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 329, total running count: 2201 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.835.596 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.837.600 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2302 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.849.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 330, total running count: 2202 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.851.106 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.852.880 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2303 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.865.127 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 331, total running count: 2203 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.866.732 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.868.338 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2304 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.880.793 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 332, total running count: 2204 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:01.882.236 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.883.877 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2305 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.896.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 333, total running count: 2205 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.898.009 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.899.571 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2306 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.912.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 334, total running count: 2206 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.913.630 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.915.332 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2307 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.927.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 335, total running count: 2207 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:01.929.015 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.931.205 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2308 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.943.458 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 336, total running count: 2208 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.944.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.946.808 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2309 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.958.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 337, total running count: 2209 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.960.281 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.962.120 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2310 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.974.532 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 338, total running count: 2210 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.975.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.977.759 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2311 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:01.990.116 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 339, total running count: 2211 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:01.991.705 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:01.994.268 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2312 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.005.864 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 340, total running count: 2212 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.007.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.009.600 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2313 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.021.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 341, total running count: 2213 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.022.928 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.025.122 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2314 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.036.989 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 342, total running count: 2214 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.038.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.040.426 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2315 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.052.250 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 343, total running count: 2215 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.053.762 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.055.913 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2316 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.067.876 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 344, total running count: 2216 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.069.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.071.341 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2317 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.083.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 345, total running count: 2217 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.085.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.086.873 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2318 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.099.264 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 346, total running count: 2218 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.100.713 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.102.353 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2319 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.114.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 347, total running count: 2219 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.116.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.119.051 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2320 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.130.531 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 348, total running count: 2220 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.131.798 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.134.426 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2321 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.145.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 349, total running count: 2221 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.147.353 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.149.876 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2322 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.161.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 350, total running count: 2222 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.163.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.165.546 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2323 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.177.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 351, total running count: 2223 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.178.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.180.316 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2324 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.192.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 352, total running count: 2224 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.194.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.195.878 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2325 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.208.522 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 353, total running count: 2225 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.209.928 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.212.571 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2326 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.223.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 354, total running count: 2226 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.225.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.227.840 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2327 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.239.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 355, total running count: 2227 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.240.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.243.297 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2328 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.254.978 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 356, total running count: 2228 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.256.398 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.258.897 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2329 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.270.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 357, total running count: 2229 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.271.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.274.417 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2330 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.286.171 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 358, total running count: 2230 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.287.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.290.034 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2331 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.301.944 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 359, total running count: 2231 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.303.204 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.305.731 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2332 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.317.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 360, total running count: 2232 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.318.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.321.566 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2333 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.332.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 361, total running count: 2233 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.334.415 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.335.919 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2334 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.348.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 362, total running count: 2234 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.349.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.351.334 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2335 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.363.962 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 363, total running count: 2235 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.365.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.366.920 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2336 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.379.154 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 364, total running count: 2236 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.380.489 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.382.794 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2337 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.394.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 365, total running count: 2237 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.396.140 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.398.510 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2338 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.410.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 366, total running count: 2238 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.411.728 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.414.082 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2339 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.425.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 367, total running count: 2239 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.427.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.429.628 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2340 batch(es) to device, channel name: 95601852-af69-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2affd0f0,python):2024-01-10-11:41:02.429.699 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:596] SendDataToAscend] ExecutionTree finished. Device queue sent number of batches: 2340 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.441.122 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 368, total running count: 2240 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.442.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.456.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 369, total running count: 2241 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.458.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.471.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 370, total running count: 2242 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.473.675 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.487.700 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 371, total running count: 2243 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.489.551 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.503.609 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 372, total running count: 2244 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.505.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.519.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 373, total running count: 2245 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.520.638 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.534.900 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 374, total running count: 2246 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.536.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.550.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 375, total running count: 2247 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.551.732 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.565.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 376, total running count: 2248 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.567.371 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.581.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 377, total running count: 2249 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.582.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.597.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 378, total running count: 2250 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.599.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.613.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 379, total running count: 2251 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.614.827 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.628.792 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 380, total running count: 2252 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.630.407 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.644.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 381, total running count: 2253 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.646.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.660.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 382, total running count: 2254 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.662.129 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.676.318 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 383, total running count: 2255 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.677.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.691.687 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 384, total running count: 2256 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.693.178 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.707.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 385, total running count: 2257 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.708.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.722.969 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 386, total running count: 2258 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.724.396 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.738.478 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 387, total running count: 2259 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.739.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.753.839 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 388, total running count: 2260 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.755.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.769.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 389, total running count: 2261 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.771.066 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.785.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 390, total running count: 2262 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.786.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.800.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 391, total running count: 2263 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.801.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.815.939 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 392, total running count: 2264 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.817.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.831.702 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 393, total running count: 2265 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.833.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.847.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 394, total running count: 2266 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.848.629 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.862.816 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 395, total running count: 2267 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.864.325 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.878.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 396, total running count: 2268 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.879.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.894.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 397, total running count: 2269 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.895.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.909.347 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 398, total running count: 2270 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.910.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.924.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 399, total running count: 2271 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.927.136 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.941.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 400, total running count: 2272 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:02.944.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.958.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 401, total running count: 2273 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.960.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:02.974.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 402, total running count: 2274 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.976.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:02.990.192 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 403, total running count: 2275 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:02.991.752 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.005.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 404, total running count: 2276 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.007.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.021.678 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 405, total running count: 2277 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.023.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.037.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 406, total running count: 2278 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.038.928 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.053.001 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 407, total running count: 2279 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.054.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.068.665 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 408, total running count: 2280 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.070.352 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.084.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 409, total running count: 2281 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.085.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.100.013 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 410, total running count: 2282 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.101.617 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.115.739 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 411, total running count: 2283 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.117.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.131.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 412, total running count: 2284 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.132.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.146.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 413, total running count: 2285 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.148.281 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.162.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 414, total running count: 2286 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.163.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.178.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 415, total running count: 2287 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.179.345 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.193.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 416, total running count: 2288 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.194.702 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.208.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 417, total running count: 2289 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.210.216 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.224.338 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 418, total running count: 2290 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.225.748 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.239.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 419, total running count: 2291 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.241.007 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.255.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 420, total running count: 2292 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.256.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.270.879 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 421, total running count: 2293 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.272.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.286.247 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 422, total running count: 2294 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.287.583 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.301.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 423, total running count: 2295 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.303.036 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.317.135 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 424, total running count: 2296 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.318.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.332.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 425, total running count: 2297 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.333.547 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.347.619 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 426, total running count: 2298 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.349.042 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.363.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 427, total running count: 2299 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.364.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.378.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 428, total running count: 2300 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.379.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.393.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 429, total running count: 2301 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.394.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.408.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 430, total running count: 2302 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.410.271 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.424.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 431, total running count: 2303 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.425.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.439.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 432, total running count: 2304 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.441.116 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.455.408 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 433, total running count: 2305 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.459.724 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.473.757 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 434, total running count: 2306 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.475.689 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.489.766 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 435, total running count: 2307 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.491.212 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.505.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 436, total running count: 2308 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.506.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.520.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 437, total running count: 2309 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.522.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.536.718 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 438, total running count: 2310 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.538.008 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.552.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 439, total running count: 2311 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.553.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.567.804 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 440, total running count: 2312 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.569.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.583.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 441, total running count: 2313 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.585.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.599.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 442, total running count: 2314 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.600.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.615.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 443, total running count: 2315 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.616.147 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.630.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 444, total running count: 2316 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.631.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.645.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 445, total running count: 2317 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.646.868 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.660.872 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 446, total running count: 2318 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.662.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.676.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 447, total running count: 2319 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.677.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.691.988 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 448, total running count: 2320 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.693.183 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.707.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 449, total running count: 2321 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.708.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.722.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 450, total running count: 2322 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.723.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.737.877 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 451, total running count: 2323 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.739.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.753.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 452, total running count: 2324 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.757.248 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.771.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 453, total running count: 2325 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.772.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.787.216 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 454, total running count: 2326 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.789.609 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.803.947 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 455, total running count: 2327 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.806.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.820.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 456, total running count: 2328 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.822.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.836.362 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 457, total running count: 2329 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.837.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.851.844 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 458, total running count: 2330 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.853.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.867.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 459, total running count: 2331 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.868.877 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.883.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 460, total running count: 2332 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.884.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.899.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 461, total running count: 2333 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.903.811 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.917.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 462, total running count: 2334 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.920.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.934.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 463, total running count: 2335 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:03.936.495 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.950.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 464, total running count: 2336 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.952.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.966.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 465, total running count: 2337 [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:03.968.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.982.132 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 466, total running count: 2338 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:03.983.863 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:03.998.207 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 467, total running count: 2339 [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:04.000.568 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:04.014.781 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 468, total running count: 2340 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:04.016.482 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_1 execution count: 5, execution time: 7301.65 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:04.016.681 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_1 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.016.785 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty epoch: 5 step: 468, loss is 0.0785270407795906 [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.017.848 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.h:76] ContinueSend] continue send at the beginning of the epoch [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.017.978 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.018.110 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.075.014 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at t10k-images-idx3-ubyte. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.075.096 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at t10k-labels-idx1-ubyte. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.075.263 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.076.980 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.077.099 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.077.124 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.077.141 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.077.178 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.077.246 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.077.272 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.077.287 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.077.307 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.077.326 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.077.381 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.077.395 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.077.419 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.077.455 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.077.850 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at t10k-images-idx3-ubyte. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.077.881 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at t10k-labels-idx1-ubyte. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.224.178 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.224.291 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.235.839 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.235.960 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.235.983 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.236.000 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.236.063 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.236.138 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.236.165 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.236.180 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.236.200 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.236.219 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.236.254 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.236.268 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.236.289 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.236.319 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.236.662 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at t10k-images-idx3-ubyte. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.236.694 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at t10k-labels-idx1-ubyte. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.365.994 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.366.111 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.376.974 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.377.112 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.377.134 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.377.157 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.377.193 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.377.266 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.377.297 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.377.317 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.377.341 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.377.362 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.377.399 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.377.418 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.377.443 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.377.477 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.377.828 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] ME(164040:281473664074816,MainProcess):2024-01-10-11:41:04.378.564 [mindspore/dataset/engine/datasets.py:4269] queue_name is newly generated. value is 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.380.358 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.380.464 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.380.490 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.380.508 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.380.541 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.380.608 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.380.638 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.380.654 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.380.677 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.380.695 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.380.732 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.380.750 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.380.770 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.380.817 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.381.132 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.381.283 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1729] InitExecDatasetVm] Start InitDataSet Entry [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:04.381.461 [mindspore/ccsrc/backend/graph_compiler/transform.cc:573] CreateBackend] CreateBackend is: ge [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.381.489 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.381.508 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEBUG(164040,ffffb1c2c440,python):2024-01-10-11:41:04.381.620 [mindspore/ccsrc/debug/debugger/debugger.cc:129] Init] Debugger got device_id: 1 [INFO] DEBUG(164040,ffffb1c2c440,python):2024-01-10-11:41:04.381.638 [mindspore/ccsrc/debug/debugger/debugger.cc:131] Init] Debugger got device_target: Ascend [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.381.770 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.381.792 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.381.804 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1252] SetRunMode] Run graph mode with kernel by kernel by configuration. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:04.381.994 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:546] CompileGraphs] Status record: start compile function graph: _anonymous__377 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.382.078 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unify_mindir_pm_0_erase_invalid_micro_depend in 3.61 us [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:04.382.176 [mindspore/ccsrc/utils/anfalgo.cc:1736] IsNodeOutputDynamicShape] Invalid base shape, node: Default/Return-op0_6 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.382.243 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.382.260 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.382.300 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.382.316 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.382.337 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.382.353 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:04.382.399 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:727] CompileGraph] Compile graph: _anonymous__377, Split segments size: 2 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:04.382.436 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:762] CompileGraphFromSegment] Compile normal segment, the first node: @_anonymous__377:CNode_378{[0]: ValueNode InitDataSetQueue} [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:04.382.498 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:421] CompileGraph] Status record: start compile graph. [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:04.382.530 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:1940] ConstructKernelGraph] Create graph: 2 [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:04.382.614 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:2697] ConstructOutput] Output:@_anonymous__377:CNode_378{[0]: ValueNode InitDataSetQueue} [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.382.876 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:193] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:2 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.382.945 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 8.84 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.035 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 61.78 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.148 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:154] CommonUnifyMindIR] start common unify mindir opt graph:2 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.194 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.274 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 75.44 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.291 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.326 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 30.88 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.344 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.359 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 14.1 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.372 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.423 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 46.77 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.545 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:79] BackendCommonOptimization] Status record: start common optimization. graph id: 2 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.593 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: add_dynamic_shape_attr [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.644 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_0_add_dynamic_shape_attr in 47.14 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.673 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.717 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_1_convert_dynamic_broadcast_to in 39.63 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.763 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_2_convert_const_input_to_attr in 26.94 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.802 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_3_custom_op_const_input_to_attr in 17.62 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.835 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 14.47 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.920 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 61.01 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.946 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 0.9 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.383.998 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_7_flatten_concat_fission in 29.73 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.041 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 20 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.083 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_9_broadcast_to_fusion in 19.94 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.126 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 20.78 us [WARNING] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.237 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:92] BackendCommonOptimization] [PROF]backend_common_optimization costs 693 usec. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.257 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:99] BackendCommonOptimization] Status record: end common optimization. graph id: 2 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.421 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_0_renorm_split in 30.38 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.444 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.522 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_1_reduce_axis_update in 71.17 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.542 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim HistogramFixedWidth not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.558 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_2_histogram_fixed_width_fusion in 15.1 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.579 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim ClipByNorm not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.593 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_3_clip_by_norm_fission in 14.98 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.621 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.635 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_4_tensor_arry_add_flow_cond1 in 15.81 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.648 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayGather not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.662 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_5_tensor_arry_add_flow_cond2 in 11.43 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.675 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.688 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_6_ge_tensor_arry_cast_index in 11.91 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.705 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArray not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.718 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_7_tensor_array_prepare in 11.97 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.730 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.765 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_8_space_to_batch_nd_attr_update in 30.92 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.781 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.810 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_9_batch_to_space_nd_attr_update in 25.79 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.825 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: avg_pool_grad_unify_mindir [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.864 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_10_avg_pool_grad_unify_mindir in 35.86 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.914 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_11_adam_weight_decay_unify_mindir in 27.8 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.384.960 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_12_cdist_fission in 22.28 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.006 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_13_cdist_grad_fission in 21.53 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.079 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_14_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 50.87 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.139 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_15_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 35.63 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.181 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_16_sparse_softmax_cross_entropy_with_logits_unify_mindir in 21.65 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.235 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_17_dropout_unify_mindir1 in 19.97 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.256 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.302 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_18_dropoutgrad_unify_mindir in 41.81 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.326 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.344 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_19_neighbor_exchange_unify_mindir in 19.95 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.358 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.375 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_20_neighbor_exchange_v2_unify_mindir in 15.41 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.391 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.404 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_21_neighbor_exchange_v2_grad_unify_mindir in 11.46 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.422 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.439 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_22_all_to_all_unify_mindir in 16.94 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.454 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim QuantDTypeCast not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.472 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_23_quant_dtype_cst_adjust in 18.12 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.486 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim FSEDecode not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.502 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_24_fse_decode_adjust in 15.39 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.519 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.533 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_25_bn_split in 12.97 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.545 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.604 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_26_bn_grad_unify_mindir in 54.42 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.625 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.656 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_27_bn_grad_split in 28.6 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.670 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.705 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_28_batchnorm_to_bninfer in 15.1 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.724 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.738 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_29_batchnormgrad_to_bninfergrad in 12.62 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.752 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.765 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_30_batch_norm_grad_infer_fission in 11.8 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.815 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_31_ascend_mindir_op_adapter in 31.96 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.836 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_32_DropoutGenMask in 1.07 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.878 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_33_lamb_fission_ge in 25.13 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.937 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_34_print_insert_placeholder_for_tensor_name in 36.9 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.385.980 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_35_getnext_for_ge in 22.92 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.386.023 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_36_sync_bn_split in 20.68 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.386.066 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_37_sync_bn_grad_split in 20.01 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.386.108 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_38_adaptive_max_pool2d_ge_fusion in 21.97 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.386.147 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_39_avg_pool_grad_for_ge in 18.02 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.386.169 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:260] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.386.276 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_40_FlashAttentionFusionV1 in 103.02 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.386.299 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:374] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.386.406 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_41_FlashAttentionFusionV1 in 104.16 us [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.386.785 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.386.816 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.386.831 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:132] GetRunMode] RunMode::kKernelMode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.386.937 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 65.86 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.387.220 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_0_seed_adapter in 57.85 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.387.254 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: env_op_attr_update [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.387.297 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_1_env_op_attr_update in 39.02 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.387.347 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_2_process call inline in 25.52 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.387.381 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_3_expander_fallback in 11.59 us [WARNING] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.387.461 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:139] GEBackendOptimizeACL] [PROF]ascend_backend_optimize_acl costs 316 usec. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:04.387.495 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] InitDataSetQueue is not defined in opdef. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.387.655 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 8.62 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.387.731 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 46.62 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.387.794 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 35.67 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.387.873 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 52.51 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.387.921 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_4_insert_type_transform_op in 19.41 us [WARNING] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.010 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:179] GEBackendOptimizeACLAfterKernelSelect] [PROF]ascend_backend_optimize_acl_after_kernel_select costs 409 usec. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.074 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_0_DropoutGenMask in 0.56 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.127 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_1_cse in 26.92 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.204 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_redundant_op in 38.34 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.234 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_3_eliminate_maketuple_getitem in 6.96 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.253 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_4_MergeTransData in 0.78 us [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.386 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive InitDataSetQueue [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.463 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive InitDataSetQueue [WARNING] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.486 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:262] CreateKernel] [PROF]create_kernel costs 109 usec. [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.526 [mindspore/ccsrc/backend/common/session/session_basic.cc:1140] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 2 start [INFO] DEBUG(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.547 [mindspore/ccsrc/debug/summary/summary.cc:33] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 2 start [INFO] DEBUG(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.567 [mindspore/ccsrc/debug/summary/summary.cc:54] RecurseSetSummaryNodesForAllGraphs] The total summary nodes is: 0 for graph: 2 [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.616 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:891] PrintGraphExecuteOrder] Graph 2 execution order: [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.664 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[0], node name[Default/InitDataSetQueue-op0_6], logic id[4294967295], stream id[0], node info[@kernel_graph_2:CNode_378{[0]: ValueNode InitDataSetQueue}] [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.696 [mindspore/ccsrc/backend/common/somas/somas.cc:143] Assign] Start Somas Assign for graph 2 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.728 [mindspore/ccsrc/backend/common/somas/somas.cc:151] Assign] Somas Configure success, configuration info: Device Name: Ascend Run by execution order: 1 Enable debug log: 0 Debug log path: . [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.745 [mindspore/ccsrc/backend/common/somas/somas.cc:154] Assign] Start Initialize SOMAS Model [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.791 [mindspore/ccsrc/backend/common/somas/somas.cc:1065] GraphOutputProcess] Set 0 graph output tensors' aligned size to 0. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.831 [mindspore/ccsrc/backend/common/somas/somas.cc:1135] RefNodeProcess] Special Tensor total size: RefNode: input 0 output 0 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.857 [mindspore/ccsrc/backend/common/somas/somas.cc:551] InitSomasModel] No Tensor from graph 2 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.872 [mindspore/ccsrc/backend/common/somas/somas.cc:157] Assign] End Initialize SOMAS Model [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.886 [mindspore/ccsrc/backend/common/somas/somas.cc:160] Assign] No Somas Tensor in graph 2 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.897 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:310] DoSomas] Somas allocate success for graph 2 somas size: 0 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.928 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:801] CreateDeviceAddress] Status record: start create device address. graph id: 2 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:04.388.987 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:808] CreateDeviceAddress] Status record: end create device address. graph id: 2 [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:04.389.021 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1129] CacheGraphOutputToFrontNodeWithIndex] Get graph backend output nodes. [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:04.389.042 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1137] CacheGraphOutputToFrontNodeWithIndex] Get graph front output nodes. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:04.389.063 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:507] CompileGraph] Status record: end compile graph. graph id: 2 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:04.389.141 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:805] CompileGraphFromSegment] Compile cut segment, the cut node: @_anonymous__377:CNode_379{[0]: ValueNode Return, [1]: CNode_378} [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:04.389.210 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:521] Transform] Graph(kernel_graph_2) transforms actor begin, strategy:pipeline_with_execution_order [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:04.389.286 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1232] BuildLoopCountActor] Create loop count actor: kernel_graph_2_LoopCountActor [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:04.389.309 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1265] BuildOutputActor] Create output actor: kernel_graph_2_OutputActor [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:04.389.326 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1285] BuildDataPrepareActor] Create data prepare actor: kernel_graph_2_DataPrepareActor [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:04.389.362 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1518] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_2 start. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:04.389.382 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1520] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_2 end. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:04.389.464 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 1_actor_set_kernel_graph_2_memory_actor_insert in 1.41 us [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:04.389.489 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 2_actor_set_kernel_graph_2_invalid_data_arrow_elimination in 1.42 us [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:04.389.521 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 3_actor_set_kernel_graph_2_multi_actor_fusion in 16.51 us [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:04.389.537 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 4_actor_set_kernel_graph_2_batch_data_arrow_fusion in 1.11 us [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:04.389.554 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:554] Transform] Graph(kernel_graph_2) transforms actor end. [WARNING] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:04.389.611 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:612] CompileGraphs] [PROF]compile_func_graph costs 7593 usec. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:04.389.632 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:615] CompileGraphs] Status record: end compile function graph: _anonymous__377, produce actor: kernel_graph_2 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:04.389.663 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_2 [INFO] GE(164040,python):2024-01-10-11:41:04.461.023 [graph_var_manager.cc:1424][EVENT]167129 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:41:04.461.125 [graph_manager.cc:1248][EVENT]167129 PreRun:PreRun start: graph node size 1, session id 31, graph id 30, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:41:04.461.798 [atrace_api.c:28](tid:167129) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:41:04.461.854 [trace_rb_log.c:84](tid:167129) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:41:04.461.868 [atrace_api.c:32](tid:167129) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:41:04.461.886 [client_manager.cpp:157][SetProfilingCallback][tid:167129] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:41:04.462.614 [parallel_partitioner.cc:165][EVENT]167129 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.462.652 [parallel_partitioner.cc:178][EVENT]167129 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.462.696 [graph_prepare.cc:1378][EVENT]167129 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.463.163 [graph_manager.cc:1050][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [481] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.463.190 [graph_manager.cc:1052][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.463.251 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [1] [INFO] GE(164040,python):2024-01-10-11:41:04.463.279 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.463.334 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [43] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.463.347 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.463.409 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.463.424 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.463.436 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.463.550 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.463.570 [graph_manager.cc:1054][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [367] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.463.817 [graph_manager.cc:1055][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [234] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.464.319 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:41:04.464.370 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.464.380 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.464.390 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [106] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.464.398 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.464.407 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:41:04.464.416 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.464.424 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.464.432 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.465.575 [graph_manager.cc:1056][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [1738] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.465.633 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.465.651 [graph_prepare.cc:1982][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [43] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.465.838 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:41:04.465.858 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [0] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.465.868 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.465.877 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferShapePass is [50] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.465.886 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [6] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.465.895 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:41:04.465.903 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [0] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.465.912 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.465.920 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of InferValuePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.465.945 [graph_prepare.cc:1983][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [282] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.465.967 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.465.978 [graph_prepare.cc:1984][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.466.002 [graph_prepare.cc:1985][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.466.017 [graph_prepare.cc:1986][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.466.028 [graph_prepare.cc:1987][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.466.042 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.466.053 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.466.066 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.466.135 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.466.147 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.466.156 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.466.165 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.466.173 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.466.182 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.466.190 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.466.198 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.466.207 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.466.215 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.466.223 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.466.231 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.466.240 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.466.248 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.466.256 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.466.265 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.466.292 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.466.304 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.466.331 [graph_prepare.cc:1988][EVENT]167129 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [295] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.466.343 [graph_manager.cc:1065][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [737] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.479.258 [graph_manager.cc:1077][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12895] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.479.308 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.479.334 [graph_manager.cc:1080][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [38] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.481.969 [graph_manager.cc:1081][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [2621] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.012 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.031 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.042 [graph_manager.cc:1082][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.071 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.084 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.099 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.132 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.145 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.158 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.170 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.202 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [23] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.220 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.236 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.255 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.279 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.292 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.302 [graph_manager.cc:2700][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [235] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.381 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of EnterPass is [0] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.482.394 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.482.404 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.482.413 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [0] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.482.421 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.482.429 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CastRemovePass is [6] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.482.438 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.482.446 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.482.455 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.482.463 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.482.471 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [6] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.482.479 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [0] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.482.488 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.482.496 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.482.504 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.482.514 [graph_manager.cc:2741][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [194] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.523 [graph_manager.cc:2752][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.546 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.558 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.579 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.594 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.606 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.617 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.639 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.653 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.666 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.677 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.695 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.707 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.719 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.732 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.740 [graph_manager.cc:2810][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [198] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.766 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.482.782 [graph_manager.cc:2821][EVENT]167129 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [29] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.815 [graph_manager.cc:1087][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [756] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.946 [graph_manager.cc:1088][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [118] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.979 [graph_manager.cc:1089][EVENT]167129 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.482.998 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.483.011 [graph_manager.cc:1097][EVENT]167129 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:41:04.483.031 [graph_manager.cc:3325][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.483.140 [engine_place.cc:144][EVENT]167129 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.483.154 [engine_place.cc:144][EVENT]167129 Run:The time cost of aicpu_ascend_kernel::CheckSupported is [16] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.483.221 [graph_manager.cc:3351][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [176] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.483.237 [graph_manager.cc:3364][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.483.301 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.483.321 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.483.426 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [95] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.483.455 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.483.494 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.483.525 [graph_manager.cc:3405][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [276] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.483.546 [graph_manager.cc:3412][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.049 [graph_manager.cc:3422][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [1487] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.085 [graph_manager.cc:3428][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.199 [graph_manager.cc:3467][EVENT]167129 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [95] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.217 [graph_manager.cc:3377][EVENT]167129 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [1969] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.232 [graph_manager.cc:1106][EVENT]167129 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [2206] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.245 [graph_manager.cc:1115][EVENT]167129 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:41:04.485.266 [graph_manager.cc:1130][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.299 [graph_manager.cc:1131][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.323 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.340 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.349 [graph_manager.cc:2837][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [35] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.405 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.485.417 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.485.426 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.485.435 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.485.444 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.485.452 [base_pass.cc:339][EVENT]167129 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:04.485.462 [graph_manager.cc:2864][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [88] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.473 [graph_manager.cc:2872][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.491 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.505 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.519 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.533 [compile_nodes_pass.cc:88][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.543 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.553 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.581 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.603 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.616 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.629 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.641 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.650 [graph_manager.cc:2927][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [161] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.664 [graph_manager.cc:2937][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.784 [graph_manager.cc:2943][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [103] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.485.804 [graph_manager.cc:2950][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.486.029 [graph_manager.cc:2958][EVENT]167129 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.486.062 [graph_manager.cc:1132][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [749] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.486.173 [graph_manager.cc:1135][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [94] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.486.209 [graph_manager.cc:2975][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.486.274 [graph_manager.cc:2981][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [54] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.486.293 [pass_manager.cc:82][EVENT]167129 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.486.307 [graph_manager.cc:2986][EVENT]167129 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.486.320 [graph_manager.cc:1136][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [126] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.486.403 [graph_manager.cc:3555][EVENT]167129 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [55] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.486.455 [engine_partitioner.cc:1139][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [11] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.486.474 [engine_partitioner.cc:1142][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.486.546 [engine_partitioner.cc:1148][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [57] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.486.572 [engine_partitioner.cc:1155][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.486.606 [engine_partitioner.cc:1164][EVENT]167129 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.486.629 [graph_builder.cc:865][EVENT]167129 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [194] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.486.698 [graph_builder.cc:288][EVENT]167129 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [49] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.486.789 [graph_builder.cc:293][EVENT]167129 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [74] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.486.975 [model_builder.cc:1133][EVENT]167129 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [97] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.487.180 [block_mem_assigner.cc:4069][EVENT]171939 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164040,python):2024-01-10-11:41:04.487.180 [block_mem_assigner.cc:4069][EVENT]171938 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164040,python):2024-01-10-11:41:04.487.472 [model_builder.cc:1144][EVENT]167129 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [473] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.487.504 [model_builder.cc:1152][EVENT]167129 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.487.522 [model_builder.cc:1157][EVENT]167129 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.487.631 [model_builder.cc:1167][EVENT]167129 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [98] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.487.654 [model_builder.cc:1174][EVENT]167129 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.487.678 [model_builder.cc:1180][EVENT]167129 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.487.718 [model_builder.cc:1184][EVENT]167129 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [25] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.487.737 [graph_builder.cc:304][EVENT]167129 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [926] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:41:04.487.907 [logger.cc:1071] 167129 ModelBindStream: model_id=576, stream_id=1857, flag=0. [INFO] GE(164040,python):2024-01-10-11:41:04.487.973 [task_generator.cc:804][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.488.021 [task_generator.cc:805][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [34] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.488.532 [task_generator.cc:814][EVENT]167129 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [497] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.488.546 [task_generator.cc:954][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [577] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.488.602 [task_generator.cc:967][EVENT]167129 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [32] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:41:04.488.620 [logger.cc:1084] 167129 ModelUnbindStream: model_id=576, stream_id=1857, [INFO] GE(164040,python):2024-01-10-11:41:04.488.678 [graph_builder.cc:310][EVENT]167129 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [928] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.488.781 [graph_manager.cc:1152][EVENT]167129 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2441] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.488.798 [graph_manager.cc:1164][EVENT]167129 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:41:04.488.829 [graph_manager.cc:1271][EVENT]167129 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [26294] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.488.840 [graph_manager.cc:1272][EVENT]167129 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:41:04.489.150 [atrace_api.c:93](tid:167129) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:41:04.489.167 [atrace_api.c:95](tid:167129) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:41:04.489.905 [model_introduction.cc:236][EVENT]167129 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(164040,python):2024-01-10-11:41:04.489.933 [model_introduction.cc:294][EVENT]167129 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(164040,python):2024-01-10-11:41:04.489.951 [model_introduction.cc:366][EVENT]167129 Data:model io_info size:0 [INFO] GE(164040,python):2024-01-10-11:41:04.491.984 [graph_converter.cc:838][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [655] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.492.049 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [19] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.492.262 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [191] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.492.326 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [41] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.492.339 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [55] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.492.363 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.492.386 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.492.405 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of ZeroCopy is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.492.438 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CEM is [24] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.492.481 [copy_flow_launch_fuse.cc:395][EVENT]167129 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.492.491 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [42] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.492.509 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.492.528 [base_optimizer.cc:70][EVENT]167129 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.492.541 [graph_converter.cc:849][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [516] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.492.659 [graph_converter.cc:853][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [108] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.493.053 [graph_converter.cc:857][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [380] micro second. [INFO] GE(164040,python):2024-01-10-11:41:04.493.125 [graph_converter.cc:862][EVENT]167129 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [51] micro second. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:04.494.371 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_2_LoopCountActor) running, loop count: 1, current count: 1, total running count: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:04.494.641 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_2 execution count: 1, execution time: 104.857 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:04.494.724 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_2 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.494.876 [mindspore/ccsrc/runtime/device/kernel_runtime_manager.cc:35] ClearGraphResource] Clear device Ascend_1 graph 2 runtime resource [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.497.103 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:198] Compile] Input plan: +-Transfer,send_epoch_end:false,total_batch:0) | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.497.232 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:216] Compile] Plan before optimization: +-Top | +-Transfer,send_epoch_end:false,total_batch:0) | | +-Repeat(count:1) | | | +-Batch(batch_size:32 drop_remainder:true) | | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | | +-MnistDataset [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.497.258 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:60] PrePass] Running pre pass loops. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.497.276 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.497.313 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.497.396 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.497.426 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.497.441 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.497.464 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.497.476 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/cache_transform_pass.cc:182] RunOnTree] Pre pass: Cache transform pass started. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.497.497 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/cache_transform_pass.cc:199] RunOnTree] Pre pass: Cache transform pass complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.497.509 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:91] PrePass] Pre pass offload complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.497.522 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:116] PostPass] Running post pass loops. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.497.551 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:135] PostPass] Post passes complete. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:04.497.587 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:230] Compile] Plan after optimization: +-Top | +-Transfer,send_epoch_end:false,total_batch:0) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.498.269 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_data_queue.cc:227] AscendTdtQueue] Select MBUF channel, the capacity of data queue is: 128 [INFO] MD(164040,fffd2a7fc0f0,python):2024-01-10-11:41:04.500.571 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at t10k-images-idx3-ubyte. [INFO] MD(164040,fffd2a7fc0f0,python):2024-01-10-11:41:04.500.608 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at t10k-labels-idx1-ubyte. [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.508.217 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:456] SendDataToAscend] Device queue, sending data to Ascend. [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.644.760 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:502] SendDataToAscend] Begin to send data to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.644.867 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:1182] PrintBeginInfoWhenFirstBatch] Loading dataset and begin to push first batch into device ... [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.645.523 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:1192] PrintEndInfoWhenFirstBatch] Loading dataset and push first batch into device successful. [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.645.548 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.646.066 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.646.541 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 3 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.646.960 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 4 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.647.407 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 5 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.647.821 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 6 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.648.231 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 7 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.648.621 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 8 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.648.974 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 9 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.649.371 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 10 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.649.771 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 11 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.650.146 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 12 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.650.625 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 13 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.651.254 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 14 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.651.629 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 15 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.651.993 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 16 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.652.328 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 17 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.652.698 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 18 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.653.085 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 19 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.653.461 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 20 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.653.876 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 21 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.654.248 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 22 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.654.609 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 23 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.654.966 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 24 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.655.325 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 25 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.655.710 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 26 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.656.066 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 27 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.656.433 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 28 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.656.797 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 29 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.657.175 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 30 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.657.507 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 31 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.657.928 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 32 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.658.290 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 33 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.658.638 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 34 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.659.009 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 35 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.659.369 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 36 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.659.712 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 37 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.660.051 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 38 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.660.399 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 39 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.660.752 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 40 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.661.138 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 41 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.661.500 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 42 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.661.926 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 43 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.662.278 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 44 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.662.622 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 45 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.662.977 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 46 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.663.324 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 47 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.663.681 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 48 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.664.027 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 49 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.664.382 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 50 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.664.733 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 51 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.665.079 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 52 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.665.460 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 53 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.665.819 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 54 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.666.172 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 55 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.666.528 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 56 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.666.877 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 57 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.667.224 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 58 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.667.589 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 59 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.667.965 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 60 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.668.328 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 61 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.668.688 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 62 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.669.060 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 63 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.669.478 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 64 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.670.008 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 65 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.670.371 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 66 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.670.771 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 67 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.671.143 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 68 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.671.505 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 69 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.671.945 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 70 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.672.318 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 71 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.672.706 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 72 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.673.146 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 73 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.673.552 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 74 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.674.129 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 75 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.674.492 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 76 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.674.883 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 77 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.675.293 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 78 batch(es) to device, channel name: 14b3c69e-af6a-11ee-be42-30fd658829ca [INFO] MD(164040,fffd2bfff0f0,python):2024-01-10-11:41:04.675.475 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:596] SendDataToAscend] ExecutionTree finished. Device queue sent number of batches: 78 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.858.382 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:978] CompileInner] Start compiling, phase: eval.1704858064700615424.281470510188880.0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.858.472 [mindspore/ccsrc/pipeline/jit/ps/event_message_print.cc:37] PrintEventMessage] Start compiling '_DataWrapper.construct' and it will take a while. Please wait... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.858.544 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1659] VmPipeline] This worker is initialized. No need to add worker action. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:04.858.578 [mindspore/ccsrc/backend/graph_compiler/transform.cc:573] CreateBackend] CreateBackend is: ge [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.858.598 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.858.612 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEBUG(164040,ffffb1c2c440,python):2024-01-10-11:41:04.858.858 [mindspore/ccsrc/debug/debugger/debugger.cc:129] Init] Debugger got device_id: 1 [INFO] DEBUG(164040,ffffb1c2c440,python):2024-01-10-11:41:04.858.873 [mindspore/ccsrc/debug/debugger/debugger.cc:131] Init] Debugger got device_target: Ascend [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.858.896 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1311] Run] Pipeline run [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.858.917 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start parse action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.860.901 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end parse action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.860.960 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start symbol_resolve action. [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.869.401 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380] Added global python symbol: {_check_is_tensor : Prim[_check_is_tensor]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.869.987 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_381{[0]: CNode_382, [1]: ValueNode logits, [2]: param_logits, [3]: CNode_383}, block: 0x333415c0/mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/nn/loss/loss.py:777/ _check_is_tensor('logits', logits, self.cls_name)/ [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.870.533 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_384{[0]: CNode_382, [1]: ValueNode labels, [2]: param_labels, [3]: CNode_385}, block: 0x333415c0/mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/nn/loss/loss.py:778/ _check_is_tensor('labels', labels, self.cls_name)/ [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.871.253 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_386{[0]: ValueNode Depend, [1]: CNode_387, [2]: CNode_388}, state: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_389{[0]: ValueNode MakeTuple, [1]: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_381{[0]: CNode_382, [1]: ValueNode logits, [2]: param_logits, [3]: CNode_383}, [2]: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_384{[0]: CNode_382, [1]: ValueNode labels, [2]: param_labels, [3]: CNode_385}} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.873.520 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_391, [1]: param_x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.873.840 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_392, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.874.119 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_393, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.874.401 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_394, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.874.671 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_395, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.874.941 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_396, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.875.210 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_397, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.875.482 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_398, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.875.744 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_399, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.876.009 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_400, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.876.270 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_401, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.876.554 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_402, [1]: x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.879.187 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_403] Added global python symbol: {len : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.879.358 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.879.714 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.879.892 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.880.390 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_403:CNode_405{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.880.530 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_404:x{[0]: CNode_406, [1]: param_фx, [2]: CNode_405} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.880.979 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_403:CNode_407{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.881.432 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_404] Added global python symbol: {len : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.881.500 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_403:CNode_408{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.881.610 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_403:x_shape{[0]: CNode_409, [1]: param_x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.881.786 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.882.090 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.882.374 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_404] Added global python symbol: {F : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.882.467 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_403] Added global python symbol: {F : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.882.539 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_403:CNode_410{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.882.857 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.887.353 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_411] Added global python symbol: {len : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.887.523 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.887.864 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.888.038 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.888.504 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_411:CNode_413{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.888.646 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_412:x{[0]: CNode_414, [1]: param_фx, [2]: CNode_413} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.889.084 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_411:CNode_415{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.889.540 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_412] Added global python symbol: {len : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.889.608 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_411:CNode_416{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.889.754 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_411:x_shape{[0]: CNode_417, [1]: param_x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.889.896 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.890.183 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.890.450 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_412] Added global python symbol: {F : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.890.559 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_411] Added global python symbol: {F : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.890.612 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_411:CNode_418{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.890.913 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.895.046 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_419] Added global python symbol: {len : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.895.214 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.895.555 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.895.743 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.896.209 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_419:CNode_421{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.896.346 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_420:x{[0]: CNode_422, [1]: param_фx, [2]: CNode_421} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.896.775 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_419:CNode_423{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.897.226 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_420] Added global python symbol: {len : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.897.292 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_419:CNode_424{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.897.397 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_419:x_shape{[0]: CNode_425, [1]: param_x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.897.528 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.897.847 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.898.115 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_420] Added global python symbol: {F : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.898.208 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_419] Added global python symbol: {F : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.898.261 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_419:CNode_426{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.898.565 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.900.699 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Flatten_construct_427] Added global python symbol: {F : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.903.674 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1261] ParseNameConstant] The NameConstant is bool:False [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.903.965 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:3 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.904.226 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.904.358 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1261] ParseNameConstant] The NameConstant is bool:True [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.905.096 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_pooling_MaxPool2d_construct_428] Added global python symbol: {isinstance : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.905.201 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_pooling_MaxPool2d_construct_429] Added global python symbol: {isinstance : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.905.258 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_pooling_MaxPool2d_construct_428 update var `isinstance` with node @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_430{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.pooling', [2]: ValueNode isinstance} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.905.417 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_pooling_MaxPool2d_construct_428] Added global python symbol: {tuple : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.905.499 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_pooling_MaxPool2d_construct_429] Added global python symbol: {tuple : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.905.562 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_pooling_MaxPool2d_construct_428 update var `tuple` with node @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_431{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.pooling', [2]: ValueNode tuple} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.905.937 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.906.055 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.906.259 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.906.365 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.906.642 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.918.488 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.918.657 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.919.233 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @canonicalize_axis_432:CNode_433{[0]: ValueNode check_axis_valid_434, [1]: param_axis, [2]: ndim}, block: 0x341e6b80/canonicalize_axis_432, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1606/ check_axis_valid(axis, ndim)/ [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.919.399 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.919.699 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @canonicalize_axis_432:CNode_435{[0]: ValueNode Depend, [1]: CNode_436, [2]: CNode_437}, state: @canonicalize_axis_432:CNode_433{[0]: ValueNode check_axis_valid_434, [1]: @canonicalize_axis_432:param_axis, [2]: @canonicalize_axis_432:ndim{[0]: CNode_438}} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.919.984 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {isinstance : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.920.137 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {Tensor : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.920.714 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {int : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.921.183 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {bool : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.921.890 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {check_flatten_order_const : Prim[check_flatten_order]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.922.388 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @2↓flatten_440:CNode_441{[0]: CNode_442, [1]: param_order}, block: 0x3165eee0/2↓flatten_440, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1615/ check_flatten_order_const(order)/ [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.922.828 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {rank_ : Prim[Rank]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.923.185 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.923.246 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.923.478 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {reshape_ : Prim[Reshape]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.923.665 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.923.968 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {ops : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.924.167 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.924.691 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {transpose_ : Prim[Transpose]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.925.119 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_443] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.925.228 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.925.297 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `shape_` with node @flatten_439:CNode_444{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode shape_} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.925.640 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_443] Added global python symbol: {rank_ : Prim[Rank]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.925.747 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `rank_` with node @flatten_439:CNode_445{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode rank_} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.926.057 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `start_dim` with node @flatten_439:param_start_dim [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.926.217 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.926.372 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `end_dim` with node @flatten_439:param_end_dim [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.926.485 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.926.760 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.926.829 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.927.060 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_443] Added global python symbol: {reshape_ : Prim[Reshape]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.927.133 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `reshape_` with node @flatten_439:CNode_446{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode reshape_} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.927.324 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.927.634 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_443] Added global python symbol: {flatten_ : Prim[Flatten]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.927.743 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {flatten_ : Prim[Flatten]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.927.812 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `flatten_` with node @flatten_439:CNode_447{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode flatten_} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.928.141 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `canonicalize_axis` with node ValueNode canonicalize_axis_432 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.928.590 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `check_dim_valid` with node ValueNode check_dim_valid_448 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.929.060 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @4↓flatten_449:CNode_450{[0]: ValueNode check_dim_valid_448, [1]: start_dim, [2]: end_dim}, block: 0x34222800/4↓flatten_449, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1636/ check_dim_valid(start_dim, end_dim)/ [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.929.301 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.929.357 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.929.632 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.930.129 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.930.689 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.931.306 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.931.770 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @2↓flatten_440:CNode_451{[0]: ValueNode Depend, [1]: CNode_452, [2]: CNode_453}, state: @2↓flatten_440:CNode_441{[0]: @flatten_439:CNode_442{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode check_flatten_order_const}, [1]: @flatten_439:param_order} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.931.903 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @4↓flatten_449:CNode_454{[0]: ValueNode Depend, [1]: CNode_455, [2]: CNode_456}, state: @4↓flatten_449:CNode_450{[0]: ValueNode check_dim_valid_448, [1]: @4↓flatten_449:idx{[0]: ValueNode canonicalize_axis_432, [1]: param_start_dim, [2]: x_rank}, [2]: @4↓flatten_449:end_dim{[0]: ValueNode canonicalize_axis_432, [1]: param_end_dim, [2]: x_rank}} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.932.017 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.932.113 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.933.270 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:174] CheckFuncReturn] Function must has 'return' statement, but missing in function 'flatten' at /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1557. FuncGraph: ↓check_dim_valid_457. We will add a 'return None' statement automatically. [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.933.442 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:174] CheckFuncReturn] Function must has 'return' statement, but missing in function 'flatten' at /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1557. FuncGraph: ↓check_axis_valid_458. We will add a 'return None' statement automatically. [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.946.033 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [shape_459] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.954.031 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end symbol_resolve action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.954.089 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start graph_reusing action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.954.107 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.basic.DenseDense[True, None]_ID [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.954.123 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.conv.Conv2dConv2dodict_values([6, 16, 5, 1, 'valid', 0, False, ])_ID [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.954.134 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.conv.Conv2dConv2dodict_values([1, 6, 5, 1, 'valid', 0, False, ])_ID [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.954.150 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end graph_reusing action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.954.168 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start meta_unpack_prepare action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.955.030 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end meta_unpack_prepare action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.955.068 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pre_cconv action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.955.085 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pre_cconv action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.955.121 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start abstract_specialize action. [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.956.669 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_463{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.956.730 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.957.137 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_464{[0]: CNode_465}, new_node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_466{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.957.192 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_464{[0]: CNode_465}, new node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_464{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.958.323 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_equal_scalar_467] Added global python symbol: {F : } [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.958.692 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 4, Shape: NoShape), AbstractScalar(Type: Int64, Value: 3, Shape: NoShape)}, g: _equal_scalar_467 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.959.358 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_468:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_468:CNode_470{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.959.427 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_468:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_468:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.961.776 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_472{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.961.837 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.962.177 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_473{[0]: CNode_474}, new_node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_475{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.962.229 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_473{[0]: CNode_474}, new node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_473{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.962.870 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_476:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_476:CNode_477{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.962.938 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_476:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_476:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.968.629 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_logical_not_scala_478] Added global python symbol: {auto_generate : } [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.969.069 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Bool, Value: true, Shape: NoShape)}, g: _logical_not_scala_478 [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.970.746 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bool_not_479] Added global python symbol: {_get_cache_prim : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.970.886 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bool_not_479] Added global python symbol: {BoolNot : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.974.515 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_480] Added global python symbol: {str : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.974.958 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↻_get_cache_prim_for_pynative_481] Added global python symbol: {str : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.975.221 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↻_get_cache_prim_for_pynative_481 update var `str` with node @↵_get_cache_prim_for_pynative_482:param_фstr [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.975.445 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_480] Added global python symbol: {tuple : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.975.646 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] _get_cache_prim_for_pynative_483 update var `key` with node @_get_cache_prim_for_pynative_483:key{[0]: CNode_484, [1]: key, [2]: CNode_485} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.976.422 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_486] Added global python symbol: {str : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.977.019 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_486] Added global python symbol: {_PRIM_CACHE : {}} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.977.112 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_480] Added global python symbol: {_PRIM_CACHE : {}} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.977.359 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_486] Added global python symbol: {Primitive : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.977.447 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_480] Added global python symbol: {Primitive : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.978.164 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @✓↓_get_cache_prim_for_pynative_487:CNode_488{[0]: ValueNode MetaFuncGraph-unpack_call.489, [1]: CNode_490, [2]: param_фargs, [3]: param_фkwargs}, block: 0x31ab6410/✓↓_get_cache_prim_for_pynative_487, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/_primitive_cache.py:84/ prim.__init__(*args, **kwargs)/ [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.978.766 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 2↓_get_cache_prim_for_pynative_491 update var `key` with node @↓_get_cache_prim_for_pynative_492:key{[0]: param_фstr, [1]: param_фkey} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.978.917 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @↻_get_cache_prim_for_pynative_493:CNode_494{[0]: ValueNode Depend, [1]: CNode_495, [2]: CNode_496}, state: @↻_get_cache_prim_for_pynative_493:CNode_497{[0]: ValueNode MetaFuncGraph-add.144, [1]: @↵_get_cache_prim_for_pynative_486:param_@CNode_497, [2]: ValueNode 1} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.979.015 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @✓↓_get_cache_prim_for_pynative_487:CNode_498{[0]: ValueNode Depend, [1]: CNode_499, [2]: CNode_500}, state: @✓↓_get_cache_prim_for_pynative_487:CNode_488{[0]: ValueNode MetaFuncGraph-unpack_call.489, [1]: @✓↓_get_cache_prim_for_pynative_487:CNode_490{[0]: ValueNode getattr, [1]: prim, [2]: ValueNode __init__}, [2]: @↵_get_cache_prim_for_pynative_486:param_фargs, [3]: @↵_get_cache_prim_for_pynative_486:param_фkwargs} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.980.376 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_501:CNode_502{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.980.442 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_501:CNode_503{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.980.486 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_501:CNode_504{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.981.126 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BoolNot. node: @bool_not_479:CNode_505{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x}, new_node: @bool_not_479:CNode_506{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.981.184 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BoolNot. node: @bool_not_479:CNode_505{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x}, new node: @bool_not_479:CNode_505{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.985.449 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_equal_string_507] Added global python symbol: {F : } [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.985.829 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: String, Value: C, Shape: NoShape), AbstractScalar(Type: String, Value: F, Shape: NoShape)}, g: _equal_string_507 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.987.312 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_508:CNode_509{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_508:CNode_510{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.987.380 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_508:CNode_509{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_508:CNode_509{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.988.631 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_neg_scalar_511] Added global python symbol: {F : } [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.988.946 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 1, Shape: NoShape)}, g: _neg_scalar_511 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.989.538 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarUsub. node: @_neg_scalar_512:CNode_513{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x}, new_node: @_neg_scalar_512:CNode_514{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.989.599 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarUsub. node: @_neg_scalar_512:CNode_513{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x}, new node: @_neg_scalar_512:CNode_513{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.990.275 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_515:CNode_516{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_515:CNode_517{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.990.357 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_515:CNode_516{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_515:CNode_516{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.990.799 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @3↓flatten_518:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput}, new_node: @3↓flatten_518:CNode_519{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.990.862 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @3↓flatten_518:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput}, new node: @3↓flatten_518:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.992.260 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_520] Added global python symbol: {F : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.992.929 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_520] Added global python symbol: {InSequence : } [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.993.284 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_520] Added global python symbol: {const_utils : } [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.993.808 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 4, Shape: NoShape), AbstractTuple{ element[0]: AbstractScalar(Type: Int64, Value: 0, Shape: NoShape), element[1]: AbstractScalar(Type: Int64, Value: 1, Shape: NoShape), sequence_nodes: {@✓3↓flatten_521:CNode_522{[0]: ValueNode MakeTuple, [1]: ValueNode 0, [2]: ValueNode 1}, elements_use_flags: {ptr: 0x33fb0630, value: [const vector]{0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: _number_in_tuple_520 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.997.126 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Flatten. node: @↓✓3↓flatten_523:CNode_524{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput}, new_node: @↓✓3↓flatten_523:CNode_525{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.997.195 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Flatten. node: @↓✓3↓flatten_523:CNode_524{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput}, new node: @↓✓3↓flatten_523:CNode_524{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.997.517 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_419:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_419:CNode_526{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.997.581 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_419:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_419:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.998.836 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_not_equal_scalar_527] Added global python symbol: {F : } [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:04.999.189 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 2, Shape: NoShape), AbstractScalar(Type: Int64, Value: 2, Shape: NoShape)}, g: _not_equal_scalar_527 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.999.893 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_528:CNode_529{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_528:CNode_530{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:04.999.961 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_528:CNode_529{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_528:CNode_529{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.001.947 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_532:CNode_533{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_532:CNode_534{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.002.020 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_532:CNode_533{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_532:CNode_533{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.003.210 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_535:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_535:CNode_536{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias, [3]: ValueNode 0} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.003.279 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_535:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_535:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias, [3]: ValueNode 0} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.003.547 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_537{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.003.611 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.003.787 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_411:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_411:CNode_538{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.003.832 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_411:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_411:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.004.697 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_539:CNode_540{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_539:CNode_541{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.004.764 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_539:CNode_540{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_539:CNode_540{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.006.596 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_542:CNode_543{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_542:CNode_544{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.006.664 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_542:CNode_543{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_542:CNode_543{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.007.831 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_545:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_545:CNode_546{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias, [3]: ValueNode 0} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.007.899 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_545:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_545:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias, [3]: ValueNode 0} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.008.170 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_547{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.008.220 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.008.397 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_403:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_403:CNode_548{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.008.442 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_403:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_403:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.009.281 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_549:CNode_550{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_549:CNode_551{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.009.347 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_549:CNode_550{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_549:CNode_550{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.011.165 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_552:CNode_553{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_552:CNode_554{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.011.236 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_552:CNode_553{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_552:CNode_553{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.012.402 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_555:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_555:CNode_556{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias, [3]: ValueNode 0} [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.012.483 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_555:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_555:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias, [3]: ValueNode 0} [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.013.377 [mindspore/ccsrc/pipeline/jit/ps/action.cc:282] AbstractAnalyze] function call depth: 0, simulate call depth: 0 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.013.511 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:222] Run] Specialize set top func graph context: {FuncGraph: mindspore_train_dataset_helper__DataWrapper_construct_460 Args: [0]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [1]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [2]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [3]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [4]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [5]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [6]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [7]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), Parent: } [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.021.132 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end abstract_specialize action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.021.190 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pack_expand action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.021.306 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pack_expand action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.021.337 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start auto_monad action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.022.401 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end auto_monad action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.022.451 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start inline action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.022.473 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end inline action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.022.508 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pre_auto_parallel action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.022.532 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pre_auto_parallel action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.022.550 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pipeline_split action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.022.564 [mindspore/ccsrc/pipeline/jit/ps/pipeline_split.cc:247] PipelineSplit] Only auto_parallel and semi_auto_parallel support pipeline split. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.022.577 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pipeline_split action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.022.594 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start optimize action. [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.025.675 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [fill_557] Added global python symbol: {cast_ : Prim[Cast]} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.025.975 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] fill_557 update var `value` with node @fill_557:value{[0]: CNode_558, [1]: param_value, [2]: param_type} [INFO] PARSER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.026.199 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [fill_557] Added global python symbol: {fillv2_ : Prim[FillV2]} [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.032.199 [mindspore/ccsrc/frontend/parallel/pynative_shard/pynative_shard.cc:334] PynativeShard] Only auto_parallel and semi_auto_parallel support pynative shard [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.032.261 [mindspore/ccsrc/frontend/parallel/step_parallel.cc:3009] StepParallel] Strategies would be ignored in data_parallel, shard() only valid in [semi_]auto_parallel. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.034.857 [mindspore/ccsrc/pipeline/jit/ps/action.cc:282] AbstractAnalyze] function call depth: 0, simulate call depth: 0 [INFO] ANALYZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.035.677 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:222] Run] Specialize set top func graph context: {FuncGraph: 382_mindspore_train_dataset_helper__DataWrapper_construct_559 Args: [0]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [1]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [2]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [3]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [4]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [5]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [6]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), [7]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x11a3c2b0, value: ValueAny), value: ValueAny), Parent: } [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.038.308 [mindspore/ccsrc/frontend/parallel/pynative_shard/pynative_shard.cc:334] PynativeShard] Only auto_parallel and semi_auto_parallel support pynative shard [INFO] OPTIMIZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.040.482 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] OPTIMIZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.040.925 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] OPTIMIZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.041.265 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.041.340 [mindspore/ccsrc/frontend/parallel/cache_embedding/cache_embedding.cc:702] AddCacheEmbedding] Parameters are all not cache enable. [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.041.823 [mindspore/ccsrc/frontend/parallel/pass/assign_add_opt.cc:120] AssignAddOpt] parallel::g_device_manager is not initialized. [INFO] OPTIMIZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.041.884 [mindspore/ccsrc/frontend/optimizer/comm_op_reuse_tag.cc:59] AddCommOpReuseTag] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.041.904 [mindspore/ccsrc/frontend/parallel/pass/overlap_opt_shard_in_pipeline.cc:70] OverlapOptShardInPipeline] parallel::g_device_manager is not initialized. [INFO] OPTIMIZER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.041.921 [mindspore/ccsrc/frontend/optimizer/grouped_pairwise_exchange_alltoall.cc:673] SetGroupedPairwiseExchangeAllToAll] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.041.942 [mindspore/ccsrc/frontend/parallel/pass/overlap_gradmatmul_and_gradallreduce.cc:358] OverlapGradMatmulAndGradAllreduce] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.041.960 [mindspore/ccsrc/frontend/parallel/pass/split_matmul_comm_elementwise_fp.cc:184] SplitMatmulCommElementwiseFp] SplitMatmulCommElementwiseFp is only support under [semi_]auto_parallel, skip it. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.041.988 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end optimize action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.042.009 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start auto_monad_reorder action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.042.127 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end auto_monad_reorder action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.042.150 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start get_jit_bprop_graph action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.042.163 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end get_jit_bprop_graph action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.042.180 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start eliminate_special_op_node action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.042.770 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end eliminate_special_op_node action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.042.828 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start validate action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.042.942 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end validate action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.042.967 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start distribtued_split action. [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.042.986 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:372] GenerateStrategy] Current parallel mode is data_parallel [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.043.000 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:384] GenerateStrategy] Generated distributed strategy is 1 [INFO] PARALLEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.043.132 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:1270] Run] All nodes are on this precoess so there's no need to build and split distributed graph. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.043.150 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end distribtued_split action. [INFO] PROFILER(164040,ffffb1c2c440,python):2024-01-10-11:41:05.043.180 [mindspore/ccsrc/plugin/device/ascend/hal/profiler/parallel_strategy_profiling.cc:48] IsProfilingParallelStrategyEnabled] Profiling parallel strategy is disabled. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.043.197 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start task_emit action. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.043.341 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.043.358 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.043.370 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1252] SetRunMode] Run graph mode with kernel by kernel by configuration. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.043.413 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:546] CompileGraphs] Status record: start compile function graph: 458_382_mindspore_train_dataset_helper__DataWrapper_construct_560 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.043.543 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unify_mindir_pm_0_erase_invalid_micro_depend in 1.8 us [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.043.954 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.043.975 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.105 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.120 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.142 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.155 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.173 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.186 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.213 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.225 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.241 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.253 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.268 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.279 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.293 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.304 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.318 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.329 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.344 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.355 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.371 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.383 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.397 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.409 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.461 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.477 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.496 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.508 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.523 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.534 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.550 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.561 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.585 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.597 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.613 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.624 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.640 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.652 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.667 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.678 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.693 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.704 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.719 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.731 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.746 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.757 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.772 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.783 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.798 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.809 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.824 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.835 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.850 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.861 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.876 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.897 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.913 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.925 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.944 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.957 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.973 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.984 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.044.999 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.045.011 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.045.025 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.045.036 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.045.082 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:727] CompileGraph] Compile graph: 458_382_mindspore_train_dataset_helper__DataWrapper_construct_560, Split segments size: 2 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.045.120 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:762] CompileGraphFromSegment] Compile normal segment, the first node: @458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:CNode_561{[0]: ValueNode Load, [1]: param_fc3.bias, [2]: ValueNode U} [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.045.305 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:421] CompileGraph] Status record: start compile graph. [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.045.342 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:1940] ConstructKernelGraph] Create graph: 3 [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.046.002 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:2697] ConstructOutput] Output:@458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:562{[0]: ValueNode Depend, [1]: 562, [2]: CNode_563} [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.046.986 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:193] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:3 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.047.485 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 36.84 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.047.770 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 235.81 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.048.190 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:154] CommonUnifyMindIR] start common unify mindir opt graph:3 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.048.603 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.048.800 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 194.64 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.048.827 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.048.866 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 34.25 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.048.885 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.048.900 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 14.09 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.048.914 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.049.107 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 185.6 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.049.547 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:79] BackendCommonOptimization] Status record: start common optimization. graph id: 3 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.050.006 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: add_dynamic_shape_attr [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.050.303 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_0_add_dynamic_shape_attr in 291.71 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.050.335 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.050.388 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_1_convert_dynamic_broadcast_to in 48.09 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.050.601 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_2_convert_const_input_to_attr in 188.11 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.050.779 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_3_custom_op_const_input_to_attr in 147.33 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.050.932 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 124.59 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.051.398 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 434.05 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.051.436 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 7.94 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.051.584 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_7_flatten_concat_fission in 122.35 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.051.722 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 112.48 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.051.871 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_9_broadcast_to_fusion in 109.4 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.052.045 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 148.37 us [WARNING] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.052.465 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:92] BackendCommonOptimization] [PROF]backend_common_optimization costs 2919 usec. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.052.498 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:99] BackendCommonOptimization] Status record: end common optimization. graph id: 3 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.110 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_0_renorm_split in 139.27 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.144 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.486 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_1_reduce_axis_update in 332.76 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.515 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim HistogramFixedWidth not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.534 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_2_histogram_fixed_width_fusion in 17.48 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.549 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim ClipByNorm not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.567 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_3_clip_by_norm_fission in 16.72 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.582 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.595 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_4_tensor_arry_add_flow_cond1 in 13.21 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.608 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayGather not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.622 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_5_tensor_arry_add_flow_cond2 in 11.79 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.635 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.648 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_6_ge_tensor_arry_cast_index in 11.38 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.661 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArray not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.674 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_7_tensor_array_prepare in 11.67 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.721 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.772 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_8_space_to_batch_nd_attr_update in 80.22 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.789 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.819 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_9_batch_to_space_nd_attr_update in 26.72 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.834 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: avg_pool_grad_unify_mindir [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.053.869 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_10_avg_pool_grad_unify_mindir in 31.5 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.054.020 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_11_adam_weight_decay_unify_mindir in 128.85 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.054.159 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_12_cdist_fission in 114.67 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.054.294 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_13_cdist_grad_fission in 110.5 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.054.470 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_14_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 147.88 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.054.661 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_15_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 159.4 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.055.356 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_16_sparse_softmax_cross_entropy_with_logits_unify_mindir in 648.56 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.055.553 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_17_dropout_unify_mindir1 in 150.17 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.055.583 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.055.747 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_18_dropoutgrad_unify_mindir in 156.78 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.055.775 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.055.793 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_19_neighbor_exchange_unify_mindir in 17.56 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.055.812 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.055.828 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_20_neighbor_exchange_v2_unify_mindir in 13.64 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.055.843 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.055.858 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_21_neighbor_exchange_v2_grad_unify_mindir in 12.61 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.055.883 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.055.900 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_22_all_to_all_unify_mindir in 14.02 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.055.916 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim QuantDTypeCast not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.055.931 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_23_quant_dtype_cst_adjust in 14.6 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.055.946 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim FSEDecode not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.055.965 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_24_fse_decode_adjust in 17.42 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.055.979 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.055.993 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_25_bn_split in 12.34 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.056.006 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.056.072 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_26_bn_grad_unify_mindir in 61.36 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.056.094 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.056.113 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_27_bn_grad_split in 17.48 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.056.131 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.056.146 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_28_batchnorm_to_bninfer in 13.34 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.056.162 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.056.176 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_29_batchnormgrad_to_bninfergrad in 12 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.056.190 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.056.204 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_30_batch_norm_grad_infer_fission in 11.95 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.056.495 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_31_ascend_mindir_op_adapter in 268.08 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.056.527 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_32_DropoutGenMask in 1.49 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.056.717 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_33_lamb_fission_ge in 155.57 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.057.068 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_34_print_insert_placeholder_for_tensor_name in 319.85 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.057.268 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_35_getnext_for_ge in 168.45 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.057.436 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_36_sync_bn_split in 136.53 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.057.610 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_37_sync_bn_grad_split in 140.63 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.057.828 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_38_adaptive_max_pool2d_ge_fusion in 184.35 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.058.003 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_39_avg_pool_grad_for_ge in 140.51 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.058.030 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:260] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.058.433 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_40_FlashAttentionFusionV1 in 398.16 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.058.465 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:374] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.058.856 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_41_FlashAttentionFusionV1 in 387.31 us [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.060.201 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.060.238 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.060.254 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:132] GetRunMode] RunMode::kKernelMode [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.060.754 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 454.92 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.061.464 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_0_seed_adapter in 430.92 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.061.498 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: env_op_attr_update [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.061.774 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_1_env_op_attr_update in 267.13 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.061.955 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_2_process call inline in 144 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.062.058 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_3_expander_fallback in 71.03 us [WARNING] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.062.311 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:139] GEBackendOptimizeACL] [PROF]ascend_backend_optimize_acl costs 1294 usec. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.062.354 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] GetNext is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.062.440 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2D is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.062.634 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPool is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.062.707 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2D is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.062.838 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPool is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.062.955 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.063.188 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.063.387 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.063.776 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] SoftmaxCrossEntropyWithLogits is not defined in opdef. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.064.238 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 70.31 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.064.659 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 373.62 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.065.043 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 343.44 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.065.628 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 540.49 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.066.181 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_4_insert_type_transform_op in 507.16 us [WARNING] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.066.437 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:179] GEBackendOptimizeACLAfterKernelSelect] [PROF]ascend_backend_optimize_acl_after_kernel_select costs 2321 usec. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.066.525 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_0_DropoutGenMask in 0.83 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.066.703 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_1_cse in 141.45 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.067.551 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_redundant_op in 812.31 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.067.743 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_3_eliminate_maketuple_getitem in 140.09 us [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.067.775 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_4_MergeTransData in 1.21 us [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.068.186 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive GetNext [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.068.377 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:467] ConvertAny] Value: ValueTuple [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.068.466 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive GetNext [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.068.496 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2D [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.068.572 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.068.673 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2D [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.068.700 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.068.778 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.068.803 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPool [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.068.898 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPool [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.068.925 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2D [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.068.983 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.069.068 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2D [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.069.094 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.069.164 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.069.189 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPool [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.069.277 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPool [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.069.303 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Flatten [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.069.373 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Flatten [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.069.408 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.069.505 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.069.530 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.069.646 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.069.674 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.069.752 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.069.776 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.069.856 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.069.882 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.069.977 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.070.003 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.070.072 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.070.097 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.070.181 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.070.207 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.070.301 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.070.326 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.070.413 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.070.438 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.070.546 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.070.570 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive OneHot [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.070.708 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive OneHot [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.070.734 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive SoftmaxCrossEntropyWithLogits [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.070.842 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive SoftmaxCrossEntropyWithLogits [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.070.870 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReduceMean [INFO] KERNEL(164040,ffffb1c2c440,python):2024-01-10-11:41:05.070.982 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReduceMean [WARNING] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.071.008 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:262] CreateKernel] [PROF]create_kernel costs 2841 usec. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.071.153 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1_6, index: 0 to input Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2_6, index: 0 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.071.191 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/Reshape-op0_6, index: 0 to input Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6, index: 0 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.071.220 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/Reshape-op1_6, index: 0 to input Default/GetNext-op1_6, index: 1 [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.071.236 [mindspore/ccsrc/backend/common/session/session_basic.cc:1140] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 3 start [INFO] DEBUG(164040,ffffb1c2c440,python):2024-01-10-11:41:05.071.250 [mindspore/ccsrc/debug/summary/summary.cc:33] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 3 start [INFO] DEBUG(164040,ffffb1c2c440,python):2024-01-10-11:41:05.071.265 [mindspore/ccsrc/debug/summary/summary.cc:54] RecurseSetSummaryNodesForAllGraphs] The total summary nodes is: 0 for graph: 3 [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.071.494 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:891] PrintGraphExecuteOrder] Graph 3 execution order: [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.071.548 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[0], node name[Default/GetNext-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:outputs{[0]: ValueNode GetNext}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.071.602 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[1], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv1-Conv2d/Conv2D-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode Conv2D, [1]: 562, [2]: CNode_564}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.071.652 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[2], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op4_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReLU, [1]: 562}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.071.694 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[3], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op3_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode MaxPool, [1]: 562}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.071.734 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[4], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/Conv2D-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode Conv2D, [1]: 562, [2]: CNode_565}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.071.771 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[5], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op5_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReLU, [1]: 562}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.071.803 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[6], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode MaxPool, [1]: 562}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.071.840 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[7], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_Flatten, [1]: 562}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.071.883 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[8], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/MatMul-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode MatMul, [1]: 562, [2]: CNode_566}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.071.930 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[9], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/BiasAdd-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_BiasAdd, [1]: 562, [2]: CNode_567, [3]: ValueNode 0}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.071.968 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[10], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op6_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReLU, [1]: 562}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.072.007 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[11], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/MatMul-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode MatMul, [1]: 562, [2]: CNode_568}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.072.056 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[12], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/BiasAdd-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_BiasAdd, [1]: 562, [2]: CNode_569, [3]: ValueNode 0}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.072.095 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[13], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op7_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReLU, [1]: 562}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.072.133 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[14], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/MatMul-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode MatMul, [1]: 562, [2]: CNode_570}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.072.169 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[15], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_BiasAdd, [1]: 562, [2]: CNode_561, [3]: ValueNode 0}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.072.221 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[16], node name[Default/Reshape-op0_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_Reshape, [1]: 562, [2]: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10])}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.072.266 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[17], node name[Default/Reshape-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_Reshape, [1]: 562, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[32])}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.072.329 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[18], node name[Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/OneHot-op0_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_OneHot, [1]: 562, [2]: ValueNode Tensor(shape=[], dtype=Int64, value=10), [3]: ValueNode Tensor(shape=[], dtype=Float32, value=1), [4]: ValueNode Tensor(shape=[], dtype=Float32, value=0), [5]: ValueNode -1}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.072.377 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[19], node name[Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/SoftmaxCrossEntropyWithLogits-op0_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode SoftmaxCrossEntropyWithLogits, [1]: 562, [2]: 562}] [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.072.428 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[20], node name[Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/ReduceMean-op0_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReduceMean, [1]: 562, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), [3]: ValueNode false}] [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.072.573 [mindspore/ccsrc/backend/common/somas/somas.cc:143] Assign] Start Somas Assign for graph 3 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.072.689 [mindspore/ccsrc/backend/common/somas/somas.cc:151] Assign] Somas Configure success, configuration info: Device Name: Ascend Run by execution order: 1 Enable debug log: 0 Debug log path: . [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.072.708 [mindspore/ccsrc/backend/common/somas/somas.cc:154] Assign] Start Initialize SOMAS Model [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.073.190 [mindspore/ccsrc/backend/common/somas/somas.cc:1065] GraphOutputProcess] Set 3 graph output tensors' aligned size to 0. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.073.379 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 7 output 8 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.073.409 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 16 output 17 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.073.427 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 1 output 18 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.073.442 [mindspore/ccsrc/backend/common/somas/somas.cc:1135] RefNodeProcess] Special Tensor total size: RefNode: input 53760 output 52608 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.073.468 [mindspore/ccsrc/backend/common/somas/somas.cc:555] InitSomasModel] Created 1 streams (0 groups), 21 nodes, 23 tensors, 3 union tensors lists, and 0 contiguous tensors lists [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.073.632 [mindspore/ccsrc/backend/common/somas/somas.cc:157] Assign] End Initialize SOMAS Model [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.073.646 [mindspore/ccsrc/backend/common/somas/somas.cc:176] Assign] Start Computing Conflict Matrix [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.073.659 [mindspore/ccsrc/backend/common/somas/somas.cc:1286] ComputeBasicMatrix] Start Conflict Computing (Bitset Model) [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.073.676 [mindspore/ccsrc/backend/common/somas/somas.cc:1291] ComputeBasicMatrix] Start Bitset [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.073.730 [mindspore/ccsrc/backend/common/somas/somas.cc:1299] ComputeBasicMatrix] Start Path Computing [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.073.746 [mindspore/ccsrc/backend/common/somas/somas.cc:1307] ComputeBasicMatrix] End Path Computing [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.073.761 [mindspore/ccsrc/backend/common/somas/somas.cc:1309] ComputeBasicMatrix] Start Tensor Relation Computing [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.073.814 [mindspore/ccsrc/backend/common/somas/somas.cc:1462] ComputeMultiTensorConflicts] Start Computing Conflicts Pairs, tensors list size is 23 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.073.840 [mindspore/ccsrc/backend/common/somas/somas.cc:1469] ComputeMultiTensorConflicts] End Computing Conflicts Pairs (time taken 0ms) [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.073.852 [mindspore/ccsrc/backend/common/somas/somas.cc:1367] ComputeBasicMatrix] End Basic Conflict Computing (Bitset Model)(time taken 0ms) [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.073.874 [mindspore/ccsrc/backend/common/somas/somas.cc:178] Assign] End Computing Conflict Matrix [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.073.886 [mindspore/ccsrc/backend/common/somas/somas.cc:1533] Solve] Somas Assign start... [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.073.906 [mindspore/ccsrc/backend/common/somas/somas.cc:1555] Solve] Start Solving [INFO] PRE_ACT(164040,fffeb67fc0f0,python):2024-01-10-11:41:05.074.037 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 20 tensors [INFO] PRE_ACT(164040,fffeb6ffd0f0,python):2024-01-10-11:41:05.074.063 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 20 tensors [INFO] PRE_ACT(164040,fffeb7fff0f0,python):2024-01-10-11:41:05.074.068 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 20 tensors [INFO] PRE_ACT(164040,fffeb77fe0f0,python):2024-01-10-11:41:05.074.055 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 20 tensors [INFO] PRE_ACT(164040,fffeb67fc0f0,python):2024-01-10-11:41:05.074.098 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 4/4 1205760 Bytes (0.00112295 GB) Single Object size(>), index(<) smallest [INFO] PRE_ACT(164040,fffeb6ffd0f0,python):2024-01-10-11:41:05.074.134 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 2/4 1205760 Bytes (0.00112295 GB) Shared Objects size(>), index(<) smallest [INFO] PRE_ACT(164040,fffeb7fff0f0,python):2024-01-10-11:41:05.074.140 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 1/4 1205760 Bytes (0.00112295 GB) Shared Objects size(>), index(<) bestfit [INFO] PRE_ACT(164040,fffeb77fe0f0,python):2024-01-10-11:41:05.074.168 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 3/4 1205760 Bytes (0.00112295 GB) Single Object size(>), index(<) bestfit [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.195 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:176] Solving] SOMAS SOLVER RESUME: [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.210 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:177] Solving] Best Solution:[1/4] [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.226 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:178] Solving] Best result:1205760 Bytes 0.00112295 GB (0.00112295 GB + 0 GB from lifelong tensors) [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.238 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:181] Solving] Best timing:0 ms [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.249 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:182] Solving] Best algorithm: Shared Objects [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.259 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:183] Solving] Best sorting strategy: size(>), index(<) [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.270 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:184] Solving] Best offset strategy: bestfit [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.281 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:185] Solving] Time elapsed: 0 ms [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.292 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:186] Solving] Spread:0 %% [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.329 [mindspore/ccsrc/backend/common/somas/somas.cc:1564] Solve] End Solving [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.357 [mindspore/ccsrc/backend/common/somas/somas.cc:2096] GenGraphStatisticInfo] Lower Bound: 1205760 (0.00112295 GB), Upper Bound: 2039296 (0.00189924 GB) [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.370 [mindspore/ccsrc/backend/common/somas/somas.cc:2099] GenGraphStatisticInfo] Total Dynamic Size (Upper Bound): 2039296 Theoretical Optimal Size (Lower Bound): 1205760 Total Workspace Size: 0 Total Communication Input Tensor Size: 0 Total Communication Output Tensor Size: 0 Total LifeLong All Tensor Size: 0 Total LifeLong Start Tensor Size: 0 Total LifeLong End Tensor Size: 2560 Reused Size(Allocate Size): 0 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.382 [mindspore/ccsrc/backend/common/somas/somas.cc:1583] Solve] Somas Assign end. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.435 [mindspore/ccsrc/backend/common/somas/somas.cc:380] UpdateSomasResultToGraph] Merged Block size: 3 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.449 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 0, offset: 602624, size: 602624 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.461 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 1, offset: 0, size: 602624 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.476 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 2, offset: 1205248, size: 512 [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.491 [mindspore/ccsrc/backend/common/somas/somas.cc:189] Assign] Somas Allocate end. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.507 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:310] DoSomas] Somas allocate success for graph 3 somas size: 1205760 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.602 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:801] CreateDeviceAddress] Status record: start create device address. graph id: 3 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.074.843 [mindspore/ccsrc/runtime/device/device_address_utils.cc:454] CreateValueNodeDeviceAddress] No device address for value node:Default/data-9_6, debug name:ValueNode U [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.075.705 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6, index is 0; cur kernel is Default/Reshape-op0_6, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.075.746 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6, index is 0; cur kernel is Default/Reshape-op0_6, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.075.775 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/GetNext-op1_6, index is 1; cur kernel is Default/Reshape-op1_6, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.075.806 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/GetNext-op1_6, index is 1; cur kernel is Default/Reshape-op1_6, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.075.827 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2_6, index is 0; cur kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1_6, index is 0 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.075.850 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2_6, index is 0; cur kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1_6, index is 0 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.075.872 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:808] CreateDeviceAddress] Status record: end create device address. graph id: 3 [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.075.958 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1129] CacheGraphOutputToFrontNodeWithIndex] Get graph backend output nodes. [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.076.033 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1137] CacheGraphOutputToFrontNodeWithIndex] Get graph front output nodes. [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.076.078 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1155] CacheGraphOutputToFrontNodeWithIndex] Backend output: Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/ReduceMean-op0_6 with index: 0 map to front node: Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits-op0_6 with index: 0 [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.076.094 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1155] CacheGraphOutputToFrontNodeWithIndex] Backend output: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6 with index: 0 map to front node: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op0_6 with index: 0 [INFO] SESSION(164040,ffffb1c2c440,python):2024-01-10-11:41:05.076.107 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1155] CacheGraphOutputToFrontNodeWithIndex] Backend output: Default/GetNext-op1_6 with index: 1 map to front node: Default/GetNext-op0_6 with index: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.076.145 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:507] CompileGraph] Status record: end compile graph. graph id: 3 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.076.421 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:805] CompileGraphFromSegment] Compile cut segment, the cut node: @458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:CNode_571{[0]: ValueNode Return, [1]: 562} [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.076.601 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:521] Transform] Graph(kernel_graph_3) transforms actor begin, strategy:pipeline_with_execution_order [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.076.659 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2619] PersistDeviceTensorForValueNode] The device address is not exist: ValueNode_572(U) [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.076.753 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1101] BuildDataSourceActor] Create queue data source actor: kernel_graph_3_DeviceDSActor_3 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.077.044 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1232] BuildLoopCountActor] Create loop count actor: kernel_graph_3_LoopCountActor [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.077.074 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1265] BuildOutputActor] Create output actor: kernel_graph_3_OutputActor [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.077.095 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1285] BuildDataPrepareActor] Create data prepare actor: kernel_graph_3_DataPrepareActor [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.077.191 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:866] CacheGraphOutputToActor] Cache graph 3 output node:Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6 debug string:@kernel_graph_3:562{[0]: ValueNode PrimFunc_BiasAdd, [1]: 562, [2]: CNode_561, [3]: ValueNode 0} with index:0 to actor:Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6, from front node:Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op0_6 debug string:@458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:562{[0]: ValueNode PrimFunc_BiasAdd, [1]: 562, [2]: CNode_561, [3]: ValueNode 0} with index:0 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.077.222 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:809] AddSomasInfoForGraphOutput] The graph 3 output node:Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6 with index: 0 somas enable or not: 1, somas offset: 545280, aligned size: 1536 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.077.307 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:866] CacheGraphOutputToActor] Cache graph 3 output node:Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/ReduceMean-op0_6 debug string:@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReduceMean, [1]: 562, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), [3]: ValueNode false} with index:0 to actor:Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/ReduceMean-op0_6, from front node:Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits-op0_6 debug string:@458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:562{[0]: ValueNode SparseSoftmaxCrossEntropyWithLogits, [1]: 562, [2]: 562} with index:0 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.077.323 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:809] AddSomasInfoForGraphOutput] The graph 3 output node:Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/ReduceMean-op0_6 with index: 0 somas enable or not: 1, somas offset: 546816, aligned size: 512 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.077.362 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:866] CacheGraphOutputToActor] Cache graph 3 output node:Default/GetNext-op1_6 debug string:@kernel_graph_3:outputs{[0]: ValueNode GetNext} with index:1 to actor:kernel_graph_3_DeviceDSActor_3, from front node:Default/GetNext-op0_6 debug string:@458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:outputs{[0]: ValueNode GetNext} with index:1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.077.383 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1518] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_3 start. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.077.429 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1520] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_3 end. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.078.163 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 1_actor_set_kernel_graph_3_memory_actor_insert in 18.54 us [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.078.200 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 2_actor_set_kernel_graph_3_invalid_data_arrow_elimination in 2.34001 us [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.078.291 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 3_actor_set_kernel_graph_3_multi_actor_fusion in 68.64 us [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.078.318 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 4_actor_set_kernel_graph_3_batch_data_arrow_fusion in 7.64 us [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.078.338 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:554] Transform] Graph(kernel_graph_3) transforms actor end. [WARNING] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.078.776 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:612] CompileGraphs] [PROF]compile_func_graph costs 35338 usec. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.078.826 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:615] CompileGraphs] Status record: end compile function graph: 458_382_mindspore_train_dataset_helper__DataWrapper_construct_560, produce actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.078.850 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end task_emit action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.078.872 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:268] SetLoopCount] Change vm_loop_flag to 0, set loop_size to 1 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.078.890 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start execute action. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.078.909 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end execute action. TotalTime = 0.220012, [19] [parse]: 0.00201645 [symbol_resolve]: 0.093112, [1] [Cycle 1]: 0.0924784, [1] [resolve]: 0.0924555 [graph_reusing]: 7.502e-05 [meta_unpack_prepare]: 0.00088763 [pre_cconv]: 4.705e-05 [abstract_specialize]: 0.0660475 [pack_expand]: 0.00013787 [auto_monad]: 0.00109692 [inline]: 5.156e-05 [pre_auto_parallel]: 3.636e-05 [pipeline_split]: 3.714e-05 [optimize]: 0.0194056, [35] [py_interpret_to_execute]: 0.00023432 [rewriter_before_opt_a]: 0.00128242 [opt_a]: 0.0148054, [2] [Cycle 1]: 0.0105014, [30] [expand_dump_flag]: 1.788e-05 [switch_simplify]: 0.0003458 [a_1]: 0.00338979 [recompute_prepare]: 3.65e-05 [updatestate_depend_eliminate]: 0.00034027 [updatestate_assign_eliminate]: 4.982e-05 [updatestate_loads_eliminate]: 0.00021867 [parameter_eliminate]: 3.27e-06 [a_2]: 0.00066557 [accelerated_algorithm]: 3.176e-05 [pynative_shard]: 4.692e-05 [auto_parallel]: 3.95e-06 [parallel]: 3.42e-05 [merge_comm]: 1.872e-05 [allreduce_fusion]: 1.164e-05 [virtual_dataset]: 1.973e-05 [get_grad_eliminate_]: 1.657e-05 [virtual_output]: 1.62e-05 [merge_forward]: 2.919e-05 [cell_reuse_recompute_pass]: 6.40001e-07 [cell_reuse_handle_not_recompute_node_pass]: 5.07e-05 [meta_fg_expand]: 2.877e-05 [after_resolve]: 2.586e-05 [a_after_grad]: 2.17e-05 [renormalize]: 0.0044151 [real_op_eliminate]: 2.531e-05 [auto_monad_grad]: 4.37001e-06 [auto_monad_eliminator]: 0.00011431 [cse]: 0.00015966 [a_3]: 0.00013855 [Cycle 2]: 0.00145964, [30] [expand_dump_flag]: 1.22e-06 [switch_simplify]: 1.809e-05 [a_1]: 0.00028872 [recompute_prepare]: 1.57e-05 [updatestate_depend_eliminate]: 3.231e-05 [updatestate_assign_eliminate]: 2.705e-05 [updatestate_loads_eliminate]: 2.591e-05 [parameter_eliminate]: 1.85e-06 [a_2]: 0.00031013 [accelerated_algorithm]: 3.002e-05 [pynative_shard]: 4.447e-05 [auto_parallel]: 3.79e-06 [parallel]: 5.95e-06 [merge_comm]: 1.198e-05 [allreduce_fusion]: 8.21e-06 [virtual_dataset]: 1.867e-05 [get_grad_eliminate_]: 1.606e-05 [virtual_output]: 1.657e-05 [merge_forward]: 2.303e-05 [cell_reuse_recompute_pass]: 3.09999e-07 [cell_reuse_handle_not_recompute_node_pass]: 4.765e-05 [meta_fg_expand]: 1.835e-05 [after_resolve]: 2.442e-05 [a_after_grad]: 2.096e-05 [renormalize]: 7.0002e-08 [real_op_eliminate]: 1.588e-05 [auto_monad_grad]: 2.09999e-06 [auto_monad_eliminator]: 6.105e-05 [cse]: 9.497e-05 [a_3]: 0.00013405 [py_interpret_to_execute_after_opt_a]: 3.373e-05 [slice_cell_reuse_recomputed_activation]: 1.85e-06 [rewriter_after_opt_a]: 0.00081712 [convert_after_rewriter]: 3.538e-05 [order_py_execute_after_rewriter]: 2.15e-05 [opt_b]: 0.00072658, [1] [Cycle 1]: 0.00072158, [7] [b_1]: 0.00045309 [b_2]: 2.058e-05 [updatestate_depend_eliminate]: 2.521e-05 [updatestate_assign_eliminate]: 2.237e-05 [updatestate_loads_eliminate]: 2.491e-05 [renormalize]: 4.015e-05 [cse]: 9.271e-05 [cconv]: 3.471e-05 [opt_after_cconv]: 0.00029577, [1] [Cycle 1]: 0.00029071, [7] [c_1]: 6.368e-05 [parameter_eliminate]: 1.48e-06 [updatestate_depend_eliminate]: 2.747e-05 [updatestate_assign_eliminate]: 2.546e-05 [updatestate_loads_eliminate]: 2.467e-05 [cse]: 8.056e-05 [renormalize]: 3.279e-05 [remove_dup_value]: 9.595e-05 [tuple_transform]: 0.00023389, [1] [Cycle 1]: 0.00022907, [3] [d_1]: 0.00013011 [d_2]: 5.063e-05 [renormalize]: 3.076e-05 [add_cache_embedding]: 6.308e-05 [add_recomputation]: 0.00027045 [cse_after_recomputation]: 0.00012866, [1] [Cycle 1]: 0.00012285, [1] [cse]: 0.00011563 [environ_conv]: 2.632e-05 [label_micro_interleaved_index]: 1.93e-06 [label_fine_grained_interleaved_index]: 1.8e-06 [assign_add_opt]: 3.332e-05 [slice_recompute_activation]: 1.48e-06 [micro_interleaved_order_control]: 1.33e-06 [full_micro_interleaved_order_control]: 1.25e-06 [comp_comm_scheduling]: 1.33e-06 [reorder_send_recv_between_fp_bp]: 1.39001e-06 [comm_op_add_attrs]: 7.80004e-07 [add_comm_op_reuse_tag]: 1.748e-05 [overlap_opt_shard_in_pipeline]: 1.402e-05 [grouped_pairwise_exchange_alltoall]: 1.32e-05 [overlap_recompute_and_grad_model_parallel]: 1.17e-06 [overlap_grad_matmul_and_grad_allreduce]: 1.286e-05 [split_matmul_comm_elemetwise]: 1.342e-05 [split_layernorm_comm]: 1.53e-06 [process_send_recv_for_ge]: 9.29998e-07 [handle_group_info]: 4.80002e-07 [auto_monad_reorder]: 0.00013281 [get_jit_bprop_graph]: 2.491e-05 [eliminate_special_op_node]: 0.00063146 [validate]: 0.00013118 [distribtued_split]: 0.00019619 [task_emit]: 0.0356662 [execute]: 3.103e-05 Sums parse : 0.002016s : 0.94% symbol_resolve.resolve : 0.092456s : 42.88% graph_reusing : 0.000075s : 0.03% meta_unpack_prepare : 0.000888s : 0.41% pre_cconv : 0.000047s : 0.02% abstract_specialize : 0.066047s : 30.63% pack_expand : 0.000138s : 0.06% auto_monad : 0.001097s : 0.51% inline : 0.000052s : 0.02% pre_auto_parallel : 0.000036s : 0.02% pipeline_split : 0.000037s : 0.02% optimize.py_interpret_to_execute : 0.000234s : 0.11% optimize.rewriter_before_opt_a : 0.001282s : 0.59% optimize.opt_a.expand_dump_flag : 0.000019s : 0.01% optimize.opt_a.switch_simplify : 0.000364s : 0.17% optimize.opt_a.a_1 : 0.003679s : 1.71% optimize.opt_a.recompute_prepare : 0.000052s : 0.02% optimize.opt_a.updatestate_depend_eliminate : 0.000373s : 0.17% optimize.opt_a.updatestate_assign_eliminate : 0.000077s : 0.04% optimize.opt_a.updatestate_loads_eliminate : 0.000245s : 0.11% optimize.opt_a.parameter_eliminate : 0.000005s : 0.00% optimize.opt_a.a_2 : 0.000976s : 0.45% optimize.opt_a.accelerated_algorithm : 0.000062s : 0.03% optimize.opt_a.pynative_shard : 0.000091s : 0.04% optimize.opt_a.auto_parallel : 0.000008s : 0.00% optimize.opt_a.parallel : 0.000040s : 0.02% optimize.opt_a.merge_comm : 0.000031s : 0.01% optimize.opt_a.allreduce_fusion : 0.000020s : 0.01% optimize.opt_a.virtual_dataset : 0.000038s : 0.02% optimize.opt_a.get_grad_eliminate_ : 0.000033s : 0.02% optimize.opt_a.virtual_output : 0.000033s : 0.02% optimize.opt_a.merge_forward : 0.000052s : 0.02% optimize.opt_a.cell_reuse_recompute_pass : 0.000001s : 0.00% optimize.opt_a.cell_reuse_handle_not_recompute_node_pass : 0.000098s : 0.05% optimize.opt_a.meta_fg_expand : 0.000047s : 0.02% optimize.opt_a.after_resolve : 0.000050s : 0.02% optimize.opt_a.a_after_grad : 0.000043s : 0.02% optimize.opt_a.renormalize : 0.004415s : 2.05% optimize.opt_a.real_op_eliminate : 0.000041s : 0.02% optimize.opt_a.auto_monad_grad : 0.000006s : 0.00% optimize.opt_a.auto_monad_eliminator : 0.000175s : 0.08% optimize.opt_a.cse : 0.000255s : 0.12% optimize.opt_a.a_3 : 0.000273s : 0.13% optimize.py_interpret_to_execute_after_opt_a : 0.000034s : 0.02% optimize.slice_cell_reuse_recomputed_activation : 0.000002s : 0.00% optimize.rewriter_after_opt_a : 0.000817s : 0.38% optimize.convert_after_rewriter : 0.000035s : 0.02% optimize.order_py_execute_after_rewriter : 0.000022s : 0.01% optimize.opt_b.b_1 : 0.000453s : 0.21% optimize.opt_b.b_2 : 0.000021s : 0.01% optimize.opt_b.updatestate_depend_eliminate : 0.000025s : 0.01% optimize.opt_b.updatestate_assign_eliminate : 0.000022s : 0.01% optimize.opt_b.updatestate_loads_eliminate : 0.000025s : 0.01% optimize.opt_b.renormalize : 0.000040s : 0.02% optimize.opt_b.cse : 0.000093s : 0.04% optimize.cconv : 0.000035s : 0.02% optimize.opt_after_cconv.c_1 : 0.000064s : 0.03% optimize.opt_after_cconv.parameter_eliminate : 0.000001s : 0.00% optimize.opt_after_cconv.updatestate_depend_eliminate : 0.000027s : 0.01% optimize.opt_after_cconv.updatestate_assign_eliminate : 0.000025s : 0.01% optimize.opt_after_cconv.updatestate_loads_eliminate : 0.000025s : 0.01% optimize.opt_after_cconv.cse : 0.000081s : 0.04% optimize.opt_after_cconv.renormalize : 0.000033s : 0.02% optimize.remove_dup_value : 0.000096s : 0.04% optimize.tuple_transform.d_1 : 0.000130s : 0.06% optimize.tuple_transform.d_2 : 0.000051s : 0.02% optimize.tuple_transform.renormalize : 0.000031s : 0.01% optimize.add_cache_embedding : 0.000063s : 0.03% optimize.add_recomputation : 0.000270s : 0.13% optimize.cse_after_recomputation.cse : 0.000116s : 0.05% optimize.environ_conv : 0.000026s : 0.01% optimize.label_micro_interleaved_index : 0.000002s : 0.00% optimize.label_fine_grained_interleaved_index : 0.000002s : 0.00% optimize.assign_add_opt : 0.000033s : 0.02% optimize.slice_recompute_activation : 0.000001s : 0.00% optimize.micro_interleaved_order_control : 0.000001s : 0.00% optimize.full_micro_interleaved_order_control : 0.000001s : 0.00% optimize.comp_comm_scheduling : 0.000001s : 0.00% optimize.reorder_send_recv_between_fp_bp : 0.000001s : 0.00% optimize.comm_op_add_attrs : 0.000001s : 0.00% optimize.add_comm_op_reuse_tag : 0.000017s : 0.01% optimize.overlap_opt_shard_in_pipeline : 0.000014s : 0.01% optimize.grouped_pairwise_exchange_alltoall : 0.000013s : 0.01% optimize.overlap_recompute_and_grad_model_parallel : 0.000001s : 0.00% optimize.overlap_grad_matmul_and_grad_allreduce : 0.000013s : 0.01% optimize.split_matmul_comm_elemetwise : 0.000013s : 0.01% optimize.split_layernorm_comm : 0.000002s : 0.00% optimize.process_send_recv_for_ge : 0.000001s : 0.00% optimize.handle_group_info : 0.000000s : 0.00% auto_monad_reorder : 0.000133s : 0.06% get_jit_bprop_graph : 0.000025s : 0.01% eliminate_special_op_node : 0.000631s : 0.29% validate : 0.000131s : 0.06% distribtued_split : 0.000196s : 0.09% task_emit : 0.035666s : 16.54% execute : 0.000031s : 0.01% Time group info: ------[substitution.] 0.090387 1195 0.01% : 0.000006s : 5: substitution.depend_value_elim 0.01% : 0.000007s : 8: substitution.float_tuple_getitem_switch 97.82% : 0.088418s : 180: substitution.getattr_setattr_resolve 0.02% : 0.000019s : 40: substitution.graph_param_transform 1.57% : 0.001417s : 75: substitution.inline 0.01% : 0.000006s : 14: substitution.less_batch_normalization 0.01% : 0.000009s : 48: substitution.load_eliminater 0.26% : 0.000239s : 428: substitution.meta_unpack_prepare 0.01% : 0.000006s : 4: substitution.minmaximum_grad 0.00% : 0.000004s : 40: substitution.partial_unused_args_eliminate 0.01% : 0.000007s : 64: substitution.remove_not_recompute_node 0.01% : 0.000006s : 16: substitution.replace_old_param 0.01% : 0.000013s : 15: substitution.switch_simplify 0.02% : 0.000022s : 8: substitution.tuple_list_convert_item_index_to_positive 0.01% : 0.000010s : 8: substitution.tuple_list_get_item_const_eliminator 0.02% : 0.000014s : 8: substitution.tuple_list_get_item_depend_reorder 0.04% : 0.000040s : 15: substitution.tuple_list_get_item_eliminator 0.02% : 0.000014s : 8: substitution.tuple_list_get_set_item_eliminator 0.06% : 0.000053s : 104: substitution.updatestate_pure_node_eliminater 0.09% : 0.000077s : 107: substitution.updatestate_useless_node_eliminater ------[renormalize.] 0.004407 2 52.38% : 0.002309s : 1: renormalize.infer 47.62% : 0.002099s : 1: renormalize.specialize ------[replace.] 0.002647 256 0.50% : 0.000013s : 2: replace.depend_value_elim 77.22% : 0.002044s : 163: replace.getattr_setattr_resolve 16.97% : 0.000449s : 75: replace.inline 5.04% : 0.000134s : 15: replace.switch_simplify 0.27% : 0.000007s : 1: replace.tuple_list_get_item_eliminator ------[match.] 0.089610 256 0.00% : 0.000001s : 2: match.depend_value_elim 98.40% : 0.088176s : 163: match.getattr_setattr_resolve 1.58% : 0.001417s : 75: match.inline 0.01% : 0.000013s : 15: match.switch_simplify 0.00% : 0.000004s : 1: match.tuple_list_get_item_eliminator ------[func_graph_cloner_run.] 0.005477 106 69.12% : 0.003786s : 29: func_graph_cloner_run.FuncGraphClonerGraph 30.88% : 0.001691s : 77: func_graph_cloner_run.FuncGraphSpecializer ------[meta_graph.] 0.000000 0 ------[manager.] 0.000000 0 ------[pynative] 0.000000 0 ------[others.] 0.099703 122 1.31% : 0.001309s : 69: opt.transform.opt_a 0.44% : 0.000441s : 23: opt.transform.opt_b 92.72% : 0.092445s : 2: opt.transform.opt_resolve 0.82% : 0.000817s : 1: opt.transforms.meta_unpack_prepare 4.39% : 0.004374s : 20: opt.transforms.opt_a 0.06% : 0.000061s : 1: opt.transforms.opt_after_cconv 0.02% : 0.000019s : 1: opt.transforms.opt_b 0.18% : 0.000178s : 2: opt.transforms.opt_trans_graph 0.06% : 0.000060s : 3: opt.transforms.special_op_eliminate [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.079.516 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1385] Run] End [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.079.552 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:846] SaveCompiledGraph] Save compiled func graph(458_382_mindspore_train_dataset_helper__DataWrapper_construct_560) phase(eval.1704858064700615424.281470510188880.0)! [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.079.572 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:864] SaveCompiledGraph] End save compiled func graph! [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.079.586 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:942] CleanCompileRes] Clean compile resource start [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.082.009 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:956] CleanCompileRes] Clean compile resource end [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.082.041 [mindspore/ccsrc/pipeline/jit/ps/event_message_print.cc:37] PrintEventMessage] End compiling '_DataWrapper.construct'. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.082.059 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1039] CompileInner] Finish compiling. [WARNING] ME(164040:281473664074816,MainProcess):2024-01-10-11:41:05.826.44 [mindspore/parallel/_utils.py:259] You are suggested to use mindspore.context.set_auto_parallel_context(parameter_broadcast=True) or mindspore.common.set_seed() to share parameters among multi-devices. [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.084.100 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.084.147 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.084.179 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.084.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Float32, value=1), output index: 0 device address:0x3435ef10 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.084.576 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Float32, value=0), output index: 0 device address:0x3321acf0 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.084.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10]), output index: 0 device address:0x3419c5a0 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.084.770 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode 1 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.084.858 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode -1 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.084.943 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Int64, value=10), output index: 0 device address:0x318e0960 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.085.024 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode 0 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.085.115 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[1], dtype=Int64, value=[32]), output index: 0 device address:0x3429ea50 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.085.208 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode false [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.085.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), output index: 0 device address:0x32fefe00 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.085.462 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] GE(164040,python):2024-01-10-11:41:05.153.513 [graph_var_manager.cc:1424][EVENT]167131 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164040,python):2024-01-10-11:41:05.153.609 [graph_manager.cc:1248][EVENT]167131 PreRun:PreRun start: graph node size 2, session id 32, graph id 31, graph name online. [INFO] ATRACE(164040,python):2024-01-10-11:41:05.154.030 [atrace_api.c:28](tid:167131) AtraceCreate start [INFO] ATRACE(164040,python):2024-01-10-11:41:05.154.088 [trace_rb_log.c:84](tid:167131) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164040,python):2024-01-10-11:41:05.154.102 [atrace_api.c:32](tid:167131) AtraceCreate end [INFO] TDT(164040,python):2024-01-10-11:41:05.154.120 [client_manager.cpp:157][SetProfilingCallback][tid:167131] [TsdClient] set profiling callback success [INFO] GE(164040,python):2024-01-10-11:41:05.154.489 [parallel_partitioner.cc:165][EVENT]167131 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.154.527 [parallel_partitioner.cc:178][EVENT]167131 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.154.573 [graph_prepare.cc:1378][EVENT]167131 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.154.702 [graph_manager.cc:1050][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [145] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.154.725 [graph_manager.cc:1052][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.154.794 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.154.823 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.154.881 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [46] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.154.894 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.154.943 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.154.955 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.154.967 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.155.097 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.155.118 [graph_manager.cc:1054][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [380] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.155.348 [graph_manager.cc:1055][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [216] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.156.175 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:41:05.156.204 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.156.215 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.156.224 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of InferShapePass is [231] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.156.233 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.156.242 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:41:05.156.250 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.156.259 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.156.268 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.157.619 [graph_manager.cc:1056][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2251] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.157.681 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.157.717 [graph_prepare.cc:1982][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [63] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.158.043 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:41:05.158.067 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.078 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.087 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of InferShapePass is [140] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.096 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.105 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [4] [INFO] GE(164040,python):2024-01-10-11:41:05.158.113 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.133 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [6] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.142 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of InferValuePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.182 [graph_prepare.cc:1983][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [451] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.158.207 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.158.220 [graph_prepare.cc:1984][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.158.233 [graph_prepare.cc:1985][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.158.248 [graph_prepare.cc:1986][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.158.259 [graph_prepare.cc:1987][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.158.274 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.158.286 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.158.299 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.158.372 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.383 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.392 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.401 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [0] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.409 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.418 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.427 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.436 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.444 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.452 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.461 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.469 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.485 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.494 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.502 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.510 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.158.532 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.158.545 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.158.574 [graph_prepare.cc:1988][EVENT]167131 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [306] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.158.587 [graph_manager.cc:1065][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [935] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.170.510 [graph_manager.cc:1077][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11903] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.170.577 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.170.629 [graph_manager.cc:1080][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [83] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.313 [graph_manager.cc:1081][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [2669] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.355 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.369 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.380 [graph_manager.cc:1082][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [33] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.408 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.423 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.436 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.468 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.482 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.494 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.517 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.554 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.572 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.599 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.622 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.635 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.647 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.656 [graph_manager.cc:2700][EVENT]167131 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [252] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.758 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.173.773 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.173.782 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.173.791 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.173.800 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.173.808 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of CastRemovePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.173.817 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.173.825 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.173.834 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.173.842 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.173.850 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [6] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.173.858 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.173.866 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [6] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.173.874 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.173.883 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.173.901 [graph_manager.cc:2741][EVENT]167131 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [226] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.911 [graph_manager.cc:2752][EVENT]167131 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.933 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.945 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.961 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.975 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.986 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.173.998 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.016 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.030 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.042 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.052 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.070 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.082 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.099 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.113 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.122 [graph_manager.cc:2810][EVENT]167131 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [193] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.147 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.174.158 [graph_manager.cc:2821][EVENT]167131 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [27] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.187 [graph_manager.cc:1087][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [790] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.317 [graph_manager.cc:1088][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [118] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.355 [graph_manager.cc:1089][EVENT]167131 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [17] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.381 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.396 [graph_manager.cc:1097][EVENT]167131 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164040,python):2024-01-10-11:41:05.174.417 [graph_manager.cc:3325][EVENT]167131 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.531 [engine_place.cc:144][EVENT]167131 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.546 [engine_place.cc:144][EVENT]167131 Run:The time cost of aicpu_ascend_kernel::CheckSupported is [42] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.614 [graph_manager.cc:3351][EVENT]167131 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [184] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.631 [graph_manager.cc:3364][EVENT]167131 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.693 [engine_partitioner.cc:1139][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.710 [engine_partitioner.cc:1142][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.831 [engine_partitioner.cc:1148][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [112] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.857 [engine_partitioner.cc:1155][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.895 [engine_partitioner.cc:1164][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [28] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.926 [graph_manager.cc:3405][EVENT]167131 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [283] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.174.943 [graph_manager.cc:3412][EVENT]167131 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.176.625 [graph_manager.cc:3422][EVENT]167131 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [1668] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.176.656 [graph_manager.cc:3428][EVENT]167131 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.176.775 [graph_manager.cc:3467][EVENT]167131 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [98] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.176.794 [graph_manager.cc:3377][EVENT]167131 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [2152] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.176.810 [graph_manager.cc:1106][EVENT]167131 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [2399] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.176.823 [graph_manager.cc:1115][EVENT]167131 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:41:05.176.845 [graph_manager.cc:1130][EVENT]167131 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.176.885 [graph_manager.cc:1131][EVENT]167131 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.176.909 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.176.926 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.176.938 [graph_manager.cc:2837][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [37] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.176.998 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [8] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.177.015 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.177.023 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of CondRemovePass is [1] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.177.032 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.177.041 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.177.049 [base_pass.cc:339][EVENT]167131 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [2] [INFO] GE(164040,python):2024-01-10-11:41:05.177.058 [graph_manager.cc:2864][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [102] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.070 [graph_manager.cc:2872][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.087 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.101 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.116 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.129 [compile_nodes_pass.cc:88][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.138 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.149 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.215 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [56] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.239 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [10] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.251 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.272 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.285 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.294 [graph_manager.cc:2927][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [209] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.306 [graph_manager.cc:2937][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.333 [graph_manager.cc:2943][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.348 [graph_manager.cc:2950][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.508 [graph_manager.cc:2958][EVENT]167131 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [32] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.542 [graph_manager.cc:1132][EVENT]167131 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [643] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.655 [graph_manager.cc:1135][EVENT]167131 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [96] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.744 [graph_manager.cc:2975][EVENT]167131 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.854 [graph_manager.cc:2981][EVENT]167131 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [94] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.870 [pass_manager.cc:82][EVENT]167131 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.881 [graph_manager.cc:2986][EVENT]167131 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.891 [graph_manager.cc:1136][EVENT]167131 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [220] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.177.989 [graph_manager.cc:3555][EVENT]167131 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [71] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.178.043 [engine_partitioner.cc:1139][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [12] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.178.062 [engine_partitioner.cc:1142][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [2] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.178.145 [engine_partitioner.cc:1148][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [74] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.178.169 [engine_partitioner.cc:1155][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [8] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.178.200 [engine_partitioner.cc:1164][EVENT]167131 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.178.231 [graph_builder.cc:865][EVENT]167131 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [212] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.178.304 [graph_builder.cc:288][EVENT]167131 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [57] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.178.447 [graph_builder.cc:293][EVENT]167131 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [125] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.178.626 [model_builder.cc:1133][EVENT]167131 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [90] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.178.893 [block_mem_assigner.cc:4069][EVENT]172170 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164040,python):2024-01-10-11:41:05.178.899 [block_mem_assigner.cc:4069][EVENT]172171 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164040,python):2024-01-10-11:41:05.179.295 [graph_mem_assigner.cc:2166][EVENT]167131 SetInputOffset:[IMAS]AfterAssignMemory : online_31 memoffset[132096], memtype[2], theory_min[264192], zero_copy[132096], total_size[132096], no_reuse[132096], streams[1], topo_mode[DFS], mop[], io_reuse[0:0], alloc_mode[] [INFO] GE(164040,python):2024-01-10-11:41:05.179.383 [model_builder.cc:1144][EVENT]167131 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [732] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.179.407 [model_builder.cc:1152][EVENT]167131 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.179.422 [model_builder.cc:1157][EVENT]167131 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.179.530 [model_builder.cc:1167][EVENT]167131 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [96] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.179.548 [model_builder.cc:1174][EVENT]167131 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [5] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.179.570 [model_builder.cc:1180][EVENT]167131 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [7] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.179.603 [model_builder.cc:1184][EVENT]167131 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [22] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.179.622 [graph_builder.cc:304][EVENT]167131 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [1151] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:41:05.179.721 [logger.cc:1071] 167131 ModelBindStream: model_id=64, stream_id=1345, flag=0. [INFO] GE(164040,python):2024-01-10-11:41:05.179.786 [task_generator.cc:804][EVENT]167131 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [4] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.179.846 [task_generator.cc:805][EVENT]167131 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [42] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.180.349 [task_generator.cc:814][EVENT]167131 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [489] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.180.363 [task_generator.cc:954][EVENT]167131 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [582] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.180.419 [task_generator.cc:967][EVENT]167131 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [33] micro second. [INFO] RUNTIME(164040,python):2024-01-10-11:41:05.180.437 [logger.cc:1084] 167131 ModelUnbindStream: model_id=64, stream_id=1345, [INFO] GE(164040,python):2024-01-10-11:41:05.180.502 [graph_builder.cc:310][EVENT]167131 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [868] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.180.606 [graph_manager.cc:1152][EVENT]167131 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2695] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.180.622 [graph_manager.cc:1164][EVENT]167131 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164040,python):2024-01-10-11:41:05.180.652 [graph_manager.cc:1271][EVENT]167131 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [26250] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.180.663 [graph_manager.cc:1272][EVENT]167131 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164040,python):2024-01-10-11:41:05.180.974 [atrace_api.c:93](tid:167131) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:41:05.180.989 [atrace_api.c:95](tid:167131) AtraceDestroy end [INFO] GE(164040,python):2024-01-10-11:41:05.181.720 [model_introduction.cc:236][EVENT]167131 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(164040,python):2024-01-10-11:41:05.181.750 [model_introduction.cc:294][EVENT]167131 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(164040,python):2024-01-10-11:41:05.181.765 [model_introduction.cc:366][EVENT]167131 Data:model io_info size:116 [INFO] GE(164040,python):2024-01-10-11:41:05.185.231 [graph_converter.cc:838][EVENT]167131 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1300] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.185.417 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of ZeroCopy is [137] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.185.873 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of CEM is [429] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.185.963 [copy_flow_launch_fuse.cc:395][EVENT]167131 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [62] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.185.980 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [79] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.186.021 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [31] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.186.051 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.186.080 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.186.152 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of CEM is [61] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.186.212 [copy_flow_launch_fuse.cc:395][EVENT]167131 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [49] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.186.222 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [60] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.186.252 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.186.277 [base_optimizer.cc:70][EVENT]167131 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.186.290 [graph_converter.cc:849][EVENT]167131 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1017] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.186.492 [graph_converter.cc:853][EVENT]167131 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [193] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.187.131 [graph_converter.cc:857][EVENT]167131 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [614] micro second. [INFO] GE(164040,python):2024-01-10-11:41:05.187.248 [graph_converter.cc:862][EVENT]167131 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [90] micro second. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.192.153 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 1 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.192.305 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 1, execution time: 108.029 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.192.449 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.192.521 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.192.554 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.192.584 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.193.890 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.193.952 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.193.992 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.194.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.196.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 2 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.196.284 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 2, execution time: 2.21926 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.196.360 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.196.405 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.196.433 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.196.456 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.197.252 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.197.304 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.197.351 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.197.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.199.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 3 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.199.615 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 3, execution time: 2.20777 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.199.691 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.199.740 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.199.771 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.199.796 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.200.576 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.200.628 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.200.661 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.200.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.202.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 4 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.202.918 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 4, execution time: 2.20335 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.202.996 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.203.041 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.203.074 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.203.116 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.203.907 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.203.958 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.204.001 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.204.153 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.206.202 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 5 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.206.291 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 5, execution time: 2.23623 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.206.367 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.206.417 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.206.446 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.206.469 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.207.244 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.207.294 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.207.324 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.207.468 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.209.370 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 6 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.209.452 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 6, execution time: 2.07575 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.209.529 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.209.576 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.209.609 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.209.636 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.210.426 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.210.478 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.210.519 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.210.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.212.612 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 7 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.212.700 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 7, execution time: 2.12618 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.212.778 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.212.825 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.212.857 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.212.887 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.213.664 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.213.722 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.213.760 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.213.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.215.791 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 8 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.215.876 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 8, execution time: 2.06238 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.215.952 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.216.006 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.216.038 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.216.062 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.216.848 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.216.899 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.216.928 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.217.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.218.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 9 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.219.060 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 9, execution time: 2.07058 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.219.137 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.219.183 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.219.210 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.219.233 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.219.999 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.220.050 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.220.079 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.220.204 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.222.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 10 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.222.249 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 10, execution time: 2.11639 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.222.324 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.222.368 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.222.396 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.222.419 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.223.197 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.223.247 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.223.276 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.223.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.225.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 11 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.225.411 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 11, execution time: 2.07156 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.225.485 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.225.530 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.225.562 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.225.588 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.226.406 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.226.456 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.226.484 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.226.627 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.228.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 12 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.228.599 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 12, execution time: 2.06296 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.228.673 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.228.716 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.228.744 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.228.767 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.229.532 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.229.582 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.229.611 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.229.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.231.686 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 13 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.231.772 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 13, execution time: 2.09337 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.231.851 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.231.900 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.231.946 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.231.973 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.232.744 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.232.794 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.232.827 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.232.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.234.831 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 14 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.234.913 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 14, execution time: 2.0297 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.234.989 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.235.037 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.235.069 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.235.092 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.235.857 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.235.907 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.235.942 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.236.089 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.238.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 15 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.238.102 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 15, execution time: 2.09347 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.238.181 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.238.228 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.238.259 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.238.281 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.239.051 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.239.101 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.239.130 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.239.278 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.241.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 16 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.241.224 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 16, execution time: 2.03701 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.241.300 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.241.347 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.241.377 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.241.404 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.242.195 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.242.246 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.242.279 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.242.420 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.244.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 17 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.244.420 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 17, execution time: 2.08714 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.244.496 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.244.543 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.244.575 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.244.598 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.245.373 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.245.423 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.245.452 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.245.606 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.247.501 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 18 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.247.580 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 18, execution time: 2.07608 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.247.656 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.247.709 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.247.738 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.247.762 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.248.529 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.248.579 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.248.608 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.248.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.250.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 19 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.250.741 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 19, execution time: 2.08107 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.250.817 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.250.865 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.250.897 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.250.919 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.251.680 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.251.730 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.251.759 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.251.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.253.762 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 20 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.253.843 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 20, execution time: 2.03299 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.253.918 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.253.966 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.253.995 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.254.021 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.254.786 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.254.837 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.254.865 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.254.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.256.876 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 21 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.256.959 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 21, execution time: 2.04065 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.257.032 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.257.077 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.257.106 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.257.129 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.257.951 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.258.002 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.258.031 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.258.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.260.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 22 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.260.141 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 22, execution time: 2.05718 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.260.215 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.260.263 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.260.297 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.260.321 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.261.083 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.261.135 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.261.164 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.261.304 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.263.168 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 23 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.263.253 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 23, execution time: 2.03552 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.263.329 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.263.378 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.263.409 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.263.432 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.264.197 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.264.248 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.264.277 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.264.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.266.375 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 24 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.266.457 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 24, execution time: 2.12801 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.266.533 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.266.581 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.266.611 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.266.636 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.267.398 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.267.449 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.267.478 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.267.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.269.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 25 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.269.546 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 25, execution time: 2.01618 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.269.619 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.269.666 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.269.720 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.269.746 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.270.508 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.270.558 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.270.587 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.270.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.272.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 26 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.272.672 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 26, execution time: 2.03263 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.272.747 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.272.791 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.272.820 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.272.843 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.273.612 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.273.663 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.273.733 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.273.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.275.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 27 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.275.864 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 27, execution time: 2.07641 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.275.936 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.275.980 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.276.010 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.276.033 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.276.796 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.276.847 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.276.876 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.277.030 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.278.989 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 28 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.279.078 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 28, execution time: 2.15016 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.279.154 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.279.208 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.279.242 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.279.266 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.280.036 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.280.088 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.280.122 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.280.260 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.282.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 29 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.282.270 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 29, execution time: 2.09789 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.282.345 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.282.393 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.282.421 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.282.449 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.283.391 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.283.442 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.283.471 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.283.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.285.480 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 30 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.285.563 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 30, execution time: 2.0398 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.285.639 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.285.717 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.285.760 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.285.787 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.286.558 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.286.608 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.286.640 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.286.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.288.627 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 31 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.288.715 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 31, execution time: 2.0175 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.288.789 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.288.837 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.288.866 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.288.888 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.289.652 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.289.712 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.289.747 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.289.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.291.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 32 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.291.846 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 32, execution time: 2.04309 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.291.920 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.291.965 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.291.994 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.292.018 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.292.802 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.292.853 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.292.881 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.293.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.294.961 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 33 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.295.061 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 33, execution time: 2.12695 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.295.136 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.295.181 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.295.208 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.295.233 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.296.135 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.296.185 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.296.213 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.296.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.298.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 34 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.298.411 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 34, execution time: 2.14543 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.298.488 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.298.535 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.298.566 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.298.589 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.299.359 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.299.409 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.299.437 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.299.584 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.301.486 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 35 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.301.580 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 35, execution time: 2.09033 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.301.654 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.301.710 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.301.747 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.301.775 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.302.542 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.302.593 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.302.623 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.302.765 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.304.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 36 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.304.746 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 36, execution time: 2.06976 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.304.823 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.304.871 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.304.903 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.304.929 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.305.736 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.305.787 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.305.818 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.305.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.307.854 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 37 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.307.942 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 37, execution time: 2.07251 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.308.016 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.308.069 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.308.102 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.308.125 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.308.891 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.308.942 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.308.974 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.309.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.311.048 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 38 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.311.133 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 38, execution time: 2.10614 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.311.208 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.311.253 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.311.280 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.311.306 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.312.077 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.312.128 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.312.157 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.312.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.314.199 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 39 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.314.285 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 39, execution time: 2.07441 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.314.358 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.314.403 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.314.431 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.314.454 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.315.215 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.315.265 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.315.293 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.315.423 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.317.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 40 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.317.328 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 40, execution time: 1.98328 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.317.403 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.317.448 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.317.476 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.317.498 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.318.304 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.318.356 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.318.388 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.318.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.320.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 41 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.320.943 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 41, execution time: 2.50053 ms in multi thread or not: 0. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.321.037 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.321.090 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.321.124 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.321.147 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.321.977 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.322.029 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.322.058 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.322.183 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.324.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 42 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.324.454 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 42, execution time: 2.33757 ms in multi thread or not: 0. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.324.535 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.324.586 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.324.617 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.324.641 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.325.423 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.325.473 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.325.503 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.325.624 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.327.808 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 43 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.327.900 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 43, execution time: 2.34151 ms in multi thread or not: 0. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.327.995 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.328.044 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.328.075 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.328.099 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.328.886 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.328.936 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.328.968 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.329.087 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.331.277 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 44 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.331.369 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 44, execution time: 2.3473 ms in multi thread or not: 0. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.331.449 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.331.502 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.331.535 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.331.559 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.332.332 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.332.382 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.332.412 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.332.532 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.334.765 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 45 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.334.860 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 45, execution time: 2.39381 ms in multi thread or not: 0. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.334.957 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.335.010 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.335.044 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.335.068 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.335.851 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.335.902 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.335.934 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.336.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.338.260 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 46 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.338.353 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 46, execution time: 2.3611 ms in multi thread or not: 0. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.338.433 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.338.481 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.338.512 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.338.535 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.339.317 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.339.369 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.339.397 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.339.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.341.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 47 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.341.788 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 47, execution time: 2.33607 ms in multi thread or not: 0. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.341.883 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.341.934 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.341.968 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.341.996 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.342.768 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.342.819 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.342.848 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.342.967 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.345.107 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 48 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.345.199 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 48, execution time: 2.2973 ms in multi thread or not: 0. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.345.280 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.345.330 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.345.361 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.345.385 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.346.235 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.346.287 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.346.317 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.346.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.348.606 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 49 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.348.701 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 49, execution time: 2.32822 ms in multi thread or not: 0. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.348.793 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.348.843 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.348.872 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.348.895 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.349.673 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.349.750 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.349.782 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.349.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.352.044 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 50 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.352.135 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 50, execution time: 2.29446 ms in multi thread or not: 0. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.352.163 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:767] SetActorExecutionStrategy] Multi thread execution time cost: 2.07294 ms, single thread execution time cost: 2.35379 ms, decide to use multi thread execution or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.352.232 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.352.279 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.352.310 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.352.333 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.353.117 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.353.167 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.353.196 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.353.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.355.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 51 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.355.317 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 51, execution time: 2.06431 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.355.395 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.355.444 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.355.474 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.355.499 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.356.269 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.356.319 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.356.350 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.356.471 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.358.433 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 52 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.358.514 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 52, execution time: 2.10692 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.358.589 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.358.637 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.358.670 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.358.693 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.359.465 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.359.517 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.359.546 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.359.696 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.361.554 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 53 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.361.655 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 53, execution time: 2.05438 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.361.755 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.361.801 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.361.829 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.361.852 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.362.624 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.362.674 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.362.705 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.362.844 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.364.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 54 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.364.849 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 54, execution time: 2.08957 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.364.923 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.364.967 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.364.995 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.365.017 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.365.830 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.365.885 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.365.917 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.366.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.367.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 55 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.367.972 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 55, execution time: 2.00008 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.368.055 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.368.103 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.368.131 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.368.158 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.368.929 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.368.980 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.369.011 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.369.140 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.371.072 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 56 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.371.150 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 56, execution time: 2.08213 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.371.227 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.371.275 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.371.307 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.371.330 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.372.106 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.372.157 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.372.188 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.372.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.374.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 57 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.374.323 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 57, execution time: 2.0792 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.374.411 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.374.455 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.374.484 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.374.507 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.375.283 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.375.333 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.375.366 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.375.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.377.378 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 58 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.377.462 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 58, execution time: 2.03994 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.377.538 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.377.583 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.377.613 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.377.637 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.378.433 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.378.483 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.378.512 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.378.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.380.485 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 59 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.380.559 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 59, execution time: 1.99273 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.380.644 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.380.690 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.380.719 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.380.742 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.381.520 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.381.570 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.381.599 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.381.737 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.383.592 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 60 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.383.667 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 60, execution time: 2.01621 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.383.741 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.383.786 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.383.815 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.383.837 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.384.607 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.384.657 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.384.691 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.384.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.386.698 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 61 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.386.779 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 61, execution time: 2.03431 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.386.863 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.386.908 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.386.937 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.386.960 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.387.732 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.387.782 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.387.811 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.387.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.389.882 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 62 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.389.963 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 62, execution time: 2.10104 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.390.037 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.390.082 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.390.112 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.390.139 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.390.904 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.390.954 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.390.984 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.391.136 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.393.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 63 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.393.130 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 63, execution time: 2.09529 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.393.218 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.393.266 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.393.295 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.393.319 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.394.098 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.394.149 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.394.180 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.394.324 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.396.218 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 64 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.396.295 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 64, execution time: 2.062 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.396.370 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.396.414 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.396.442 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.396.465 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.397.239 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.397.288 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.397.317 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.397.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.399.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 65 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.399.456 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 65, execution time: 2.08813 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.399.541 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.399.588 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.399.616 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.399.640 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.400.400 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.400.449 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.400.478 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.400.620 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.402.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 66 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.402.610 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 66, execution time: 2.07908 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.402.687 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.402.733 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.402.761 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.402.784 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.403.557 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.403.607 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.403.637 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.403.776 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.405.739 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 67 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.405.816 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 67, execution time: 2.12294 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.405.891 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.405.946 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.405.975 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.405.997 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.406.766 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.406.816 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.406.848 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.406.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.408.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 68 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.408.939 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 68, execution time: 2.03896 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.409.011 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.409.055 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.409.084 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.409.108 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.409.901 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.409.951 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.409.984 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.410.130 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.412.048 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 69 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.412.129 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 69, execution time: 2.08905 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.412.203 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.412.257 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.412.285 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.412.307 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.413.070 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.413.121 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.413.153 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.413.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.415.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 70 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.415.337 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 70, execution time: 2.12868 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.415.412 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.415.457 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.415.486 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.415.510 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.416.273 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.416.324 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.416.355 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.416.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.418.415 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 71 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.418.501 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 71, execution time: 2.09156 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.418.579 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.418.632 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.418.659 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.418.681 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.419.443 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.419.496 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.419.527 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.419.677 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.421.567 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 72 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.421.637 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 72, execution time: 2.05838 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.421.729 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.421.779 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.421.809 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.421.835 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.422.599 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.422.649 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.422.680 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.422.822 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.424.736 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 73 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.424.808 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 73, execution time: 2.07588 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.424.880 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.424.942 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.424.973 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.424.996 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.425.797 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.425.847 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.425.877 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.426.020 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.427.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 74 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.427.956 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 74, execution time: 2.02705 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.428.033 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.428.082 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.428.109 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.428.134 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.428.898 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.428.946 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.428.978 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.429.109 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.431.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 75 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.431.127 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 75, execution time: 2.09313 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.431.202 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.431.249 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.431.288 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.431.311 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.432.083 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.432.133 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.432.165 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4a7440f0,python):2024-01-10-11:41:05.432.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.434.214 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 76 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.434.296 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 76, execution time: 2.07651 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.434.369 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.434.413 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.434.442 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.434.464 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.435.228 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.435.278 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.435.306 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.435.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe4af450f0,python):2024-01-10-11:41:05.437.345 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 77 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.437.424 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 77, execution time: 2.06661 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.437.498 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.437.542 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.437.580 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.437.605 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164040,ffffb1c2c440,python):2024-01-10-11:41:05.438.414 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.438.466 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.438.494 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164040,fffe4b7460f0,python):2024-01-10-11:41:05.438.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164040,fffe49f430f0,python):2024-01-10-11:41:05.440.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 78 [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:05.440.609 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 78, execution time: 2.06312 ms in multi thread or not: 1. [INFO] VM(164040,ffffb1c2c440,python):2024-01-10-11:41:05.440.681 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.440.725 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.440.753 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.440.775 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty Accuracy: 0.07491987179487179 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.719.160 [mindspore/ccsrc/pipeline/jit/ps/init.cc:515] operator()] Start register... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.719.249 [mindspore/ccsrc/pipeline/jit/ps/init.cc:519] operator()] Start mindspore.profiler... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.719.326 [mindspore/ccsrc/pipeline/jit/ps/init.cc:527] operator()] Start EmbeddingCacheScheduler... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.719.351 [mindspore/ccsrc/pipeline/jit/ps/init.cc:534] operator()] Start releasing dataset handles... [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:05.719.407 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164040,ffffb1c2c440,python):2024-01-10-11:41:05.719.508 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.726.617 [mindspore/ccsrc/pipeline/jit/ps/init.cc:537] operator()] End release dataset handles. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:05.726.668 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2393] FinalizeCluster] Start finalize the cluster instance. [INFO] DISTRIBUTED(164040,ffff23fff0f0,python):2024-01-10-11:41:06.647.084 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:301] Heartbeat] The heartbeat thread is finished. [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:41:06.647.499 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:131] Finalize] The compute graph node has been unregistered successfully. [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:41:06.647.648 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:533] Finalize] Delete send event loop [INFO] DISTRIBUTED(164040,ffff319030f0,python):2024-01-10-11:41:06.647.749 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:41:06.647.925 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:209] Finalize] Stop loop succ [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:41:06.647.946 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:540] Finalize] Delete recv event loop [INFO] DISTRIBUTED(164040,ffff321040f0,python):2024-01-10-11:41:06.648.011 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:41:06.648.149 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:209] Finalize] Stop loop succ [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:41:06.648.165 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:554] Finalize] Delete connection pool. [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:41:06.648.222 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:533] Finalize] Delete send event loop [INFO] DISTRIBUTED(164040,ffff309010f0,python):2024-01-10-11:41:06.648.286 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:41:06.648.435 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:209] Finalize] Stop loop succ [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:41:06.648.452 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:540] Finalize] Delete recv event loop [INFO] DISTRIBUTED(164040,ffff311020f0,python):2024-01-10-11:41:06.648.488 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:41:06.648.603 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:209] Finalize] Stop loop succ [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:41:06.648.618 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:554] Finalize] Delete connection pool. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:06.648.635 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2396] FinalizeCluster] End finalize the cluster instance. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:06.648.649 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2335] ClearResAtexit] Pipeline clear all resource [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:06.648.742 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:290] RecordExitStatus] Status record: system exit. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:06.652.954 [mindspore/ccsrc/runtime/graph_scheduler/rpc_node_scheduler.cc:220] Clear] Start finalizing tcp server and client for rpc actors. [INFO] RUNTIME_FRAMEWORK(164040,ffffb1c2c440,python):2024-01-10-11:41:06.652.988 [mindspore/ccsrc/runtime/graph_scheduler/rpc_node_scheduler.cc:230] Clear] End finalizing tcp server and client for rpc actors. [INFO] ME(164040,ffffb1c2c440,python):2024-01-10-11:41:06.653.351 [mindspore/core/mindrt/src/actor/actormgr.cc:153] Finalize] mindrt Actors finish exiting. [INFO] ME(164040,ffffb1c2c440,python):2024-01-10-11:41:06.653.372 [mindspore/core/mindrt/src/actor/actormgr.cc:156] Finalize] mindrt Threads finish exiting. [INFO] ME(164040,ffffb1c2c440,python):2024-01-10-11:41:06.669.466 [mindspore/core/mindrt/src/actor/actormgr.cc:167] Finalize] mindrt IOMGRS finish exiting. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:06.670.280 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2207] ClearResPart1] Start Finalize StreamSynchronizer... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:06.670.319 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2209] ClearResPart1] End Finalize StreamSynchronizer... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:06.671.649 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:829] ClearRes] Clean executor resource! [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:06.671.679 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2223] ClearResPart2] Start clear PyNativeExecutor... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:06.671.998 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2225] ClearResPart2] End clear PyNativeExecutor. [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:41:06.672.048 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:179] ClearGraph] Remove all graphs in GraphManager [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:06.676.250 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2241] ClearResPart2] Start clear kernel runtime... [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:06.676.290 [mindspore/ccsrc/runtime/device/kernel_runtime_manager.cc:25] ClearRuntimeResource] Release device Ascend_1 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:06.676.306 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:240] ReleaseDeviceRes] Ascend finalize start [INFO] HCCL(164040,python):2024-01-10-11:41:06.676.396 [op_base.cc:1312][164040]com is not global com [INFO] HCCP(164040,python):2024-01-10-11:41:06.676.581 [ra_host.c:863]tid:164040,ra_socket_batch_close(863) : Input parameters: [0]th, phy_id[1], local_ip[1.0.0.0] [INFO] HCCP(164040,python):2024-01-10-11:41:06.676.885 [ra_host.c:1795]tid:164040,ra_socket_white_list_del(1795) : Input parameters: phy_id[1], local_ip[1.0.0.0], num[1] [INFO] HCCP(164040,python):2024-01-10-11:41:06.677.363 [ra_host.c:863]tid:164040,ra_socket_batch_close(863) : Input parameters: [0]th, phy_id[1], local_ip[1.0.0.0] [INFO] HCCP(164040,python):2024-01-10-11:41:06.778.735 [ra_host.c:863]tid:164040,ra_socket_batch_close(863) : Input parameters: [0]th, phy_id[1], local_ip[1.0.0.0] [INFO] HCCP(164040,python):2024-01-10-11:41:06.778.766 [ra_host.c:863]tid:164040,ra_socket_batch_close(863) : Input parameters: [1]th, phy_id[1], local_ip[1.0.0.0] [INFO] HCCP(164040,python):2024-01-10-11:41:06.778.778 [ra_host.c:863]tid:164040,ra_socket_batch_close(863) : Input parameters: [2]th, phy_id[1], local_ip[1.0.0.0] [INFO] HCCP(164040,python):2024-01-10-11:41:06.825.091 [ra_host.c:1795]tid:164040,ra_socket_white_list_del(1795) : Input parameters: phy_id[1], local_ip[1.0.0.0], num[1] [INFO] HCCP(164040,python):2024-01-10-11:41:06.829.249 [ra_host.c:941]tid:164040,ra_socket_listen_stop(941) : Input parameters: [0]th, phy_id[1], local_ip[1.0.0.0] [INFO] HCCP(164040,python):2024-01-10-11:41:06.829.394 [ra_host.c:525]tid:164040,ra_socket_deinit(525) : Input parameters: phy_id[1] family[2] local_ip[1.0.0.0] [INFO] HCCP(164040,python):2024-01-10-11:41:06.829.477 [ra_host.c:349]tid:164040,ra_deinit(349) : Input parameters: phy_id[1], nic_position:[1] [INFO] HCCP(164040,python):2024-01-10-11:41:06.829.489 [ra_hdc.c:1535]tid:164040,ra_hdc_deinit(1535) : hdc deinit start! phy_id[1] [INFO] HCCP(164040,python):2024-01-10-11:41:06.829.582 [ra_hdc.c:1570]tid:164040,ra_hdc_deinit(1570) : hdc deinit OK! phy_id[1] [INFO] ATRACE(164040,python):2024-01-10-11:41:06.829.683 [atrace_api.c:93](tid:164040) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:41:06.829.732 [atrace_api.c:95](tid:164040) AtraceDestroy end [INFO] HCCL(164040,python):2024-01-10-11:41:06.852.251 [op_base.cc:1332][164040]op_base comm destroy complete,take time [175883]us, rankNum[0], rank[4294967295], deviceLogicId[1] [INFO] HCCL_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:41:06.852.317 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:260] FinalizeHccl] Start destroy hccl adapter for GRAPH_MODE [INFO] HCCL_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:41:06.852.345 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:548] FinalizeHcclExec] Start finalize hccl exec. [INFO] HCCL(164040,python):2024-01-10-11:41:06.856.463 [hcom_executor.cc:43][164040][Finalize][HcomExecutor]Hcom Excutor Finalize end. ret[0] [INFO] HCCL_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:41:06.856.523 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:556] FinalizeHcclExec] HcclExec destroy success [INFO] HCCL_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:41:06.856.548 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:388] FinalizeKernelInfoStore] Start destroy hccl kernel info store. [INFO] HCCL_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:41:06.856.608 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:410] FinalizeKernelInfoStore] Destroy hccl kernel info store success. [INFO] HCCL_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:41:06.856.624 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:437] FinalizeHcclComm] Start finalize hccl comm. [INFO] HCCL_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:41:06.856.736 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:273] FinalizeHccl] Destroy hccl adapter success. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:06.856.751 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:710] DestroyHccl] Hccl destroy successful. [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:06.856.803 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:783] operator()] Common mem pool info: Total allocated mem:1024M, peak used mem:4M, in used mem:0M, total idle mem:1023M. Block unit size:1024M, block counts:1, block[0] block size:1024M idle size:1023M [INFO] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:06.856.832 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:783] operator()] Persistent mem pool info: Total allocated mem:1024M, peak used mem:0M, in used mem:0M, total idle mem:1023M. Block unit size:1024M, block counts:1, block[0] block size:1024M idle size:1023M [WARNING] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:06.856.847 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:793] DumpDynamicMemPoolStateInfo] The dynamic memory pool total allocated mem:2048M, peak used mem:5M, in used mem:0M, total idle mem:71812935M, total eager free mem:0M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [WARNING] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:08.384.315 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:142] DeInitialize] Ascend Memory Adapter deinitialize success, statistics: Device HBM memory size: 32768M MindSpore Used memory size: 30680M MindSpore memory base address: 0x124100000000 Total Static Memory size: 2048M Total Dynamic memory size: 0M Dynamic memory size of this graph: 0M [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:08.404.281 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:274] ReleaseDeviceRes] Ascend finalize end [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:08.404.338 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2243] ClearResPart2] End clear kernel runtime. [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:41:08.404.359 [mindspore/ccsrc/distributed/collective/collective_manager.cc:379] Finalize] Begin finalize collective manager. [INFO] DISTRIBUTED(164040,fffed8ff90f0,python):2024-01-10-11:41:08.404.494 [mindspore/ccsrc/distributed/collective/collective_manager.cc:358] operator()] Start finalizing host communication lib. [INFO] DISTRIBUTED(164040,fffed8ff90f0,python):2024-01-10-11:41:08.404.522 [mindspore/ccsrc/distributed/collective/collective_manager.cc:362] operator()] End finalizing host communication lib. [INFO] DISTRIBUTED(164040,fffed8ff90f0,python):2024-01-10-11:41:08.404.535 [mindspore/ccsrc/distributed/collective/collective_manager.cc:367] operator()] Start finalizing device communication lib. [INFO] DISTRIBUTED(164040,fffed8ff90f0,python):2024-01-10-11:41:08.404.546 [mindspore/ccsrc/distributed/collective/collective_manager.cc:371] operator()] End finalizing device communication lib. [INFO] DISTRIBUTED(164040,ffffb1c2c440,python):2024-01-10-11:41:08.404.596 [mindspore/ccsrc/distributed/collective/collective_manager.cc:386] Finalize] End finalize collective manager. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:08.404.612 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2258] ClearResPart2] Start clear device context... [INFO] ME(164040,ffffb1c2c440,python):2024-01-10-11:41:08.404.626 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:469] ClearDeviceContexts] Release device Ascend_1 [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:41:08.404.650 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:264] DeleteGraphRunner] Delete GraphRunner success [TRACE] GE(164040,python):2024-01-10-11:41:08.404.669 [status:INIT] [ge_api.cc:463]164040 ~Session:Start to destruct session. [TRACE] GE(164040,python):2024-01-10-11:41:08.404.699 [status:RUNNING] [ge_api.cc:475]164040 ~Session:Session id is 0 [TRACE] GE(164040,python):2024-01-10-11:41:08.404.710 [status:RUNNING] [ge_api.cc:476]164040 ~Session:Destroying session [TRACE] GE(164040,python):2024-01-10-11:41:08.405.554 [status:STOP] [ge_api.cc:491]164040 ~Session:Session Destructor finished [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:41:08.405.593 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:235] DeleteGeSession] Delete Ge Session success [TRACE] GE(164040,python):2024-01-10-11:41:08.405.613 [status:INIT] [ge_api.cc:301]164040 GEFinalize:GEFinalize start [INFO] GE(164040,python):2024-01-10-11:41:08.405.754 [execution_runtime.cc:80][EVENT]164040 FinalizeExecutionRuntime:Execution runtime finalize begin. [INFO] GE(164040,python):2024-01-10-11:41:08.405.774 [execution_runtime.cc:92][EVENT]164040 FinalizeExecutionRuntime:Execution runtime finalized. [TRACE] GE(164040,python):2024-01-10-11:41:08.405.785 [status:RUNNING] [ge_api.cc:313]164040 GEFinalize:Finalizing environment [INFO] TUNE(164040,python):2024-01-10-11:41:08.735.899 [cann_kb_pyfunc_mgr.cpp:127][CANNKB][Tid:164040]"CannKbPyfuncMgr: enter PyObjectDeinit function, reference_[1]" [INFO] TUNE(164040,python):2024-01-10-11:41:08.735.955 [cann_kb_pyfunc_mgr.cpp:138][CANNKB][Tid:164040]"CannKbPyfuncMgr: PyObjectDeinit function end successfully!" [INFO] GE(164040,python):2024-01-10-11:41:08.766.216 [gelib.cc:324][EVENT]164040 SystemFinalize:Online infer finalize GELib success. [TRACE] GE(164040,python):2024-01-10-11:41:09.113.107 [status:STOP] [ge_api.cc:341]164040 GEFinalize:GEFinalize finished [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.113.201 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:317] CloseTsd] Start to close tsd, ref = 1 [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.113.817 [mindspore/ccsrc/plugin/device/ascend/hal/device/tensorprint_utils.cc:449] DestroyTensorPrintThread] Succeed stop acl data channel for host queue [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.114.251 [mindspore/ccsrc/plugin/device/ascend/hal/device/tensorprint_utils.cc:375] JoinAclPrintThread] join acl tdt host receive process [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.114.310 [mindspore/ccsrc/plugin/device/ascend/hal/device/tensorprint_utils.cc:463] DestroyTensorPrintThread] Succeed destroy acl channel [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.114.341 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:119] DestoryHandler] The thread of ms_histogram_summary channel is being destroyed. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.114.355 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:115] ~MbufDataHandler] Channel ms_histogram_summary begins the destruction process. [INFO] DEVICE(164040,fffee95850f0,python):2024-01-10-11:41:09.198.155 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:67] ~ScopeAclTdtDataset] AcltdtDestroyDataset succeed. [INFO] DEVICE(164040,fffedaffd0f0,python):2024-01-10-11:41:09.198.235 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:67] ~ScopeAclTdtDataset] AcltdtDestroyDataset succeed. [INFO] DEVICE(164040,fffee8d840f0,python):2024-01-10-11:41:09.198.255 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:67] ~ScopeAclTdtDataset] AcltdtDestroyDataset succeed. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.198.742 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:119] DestoryHandler] The thread of ms_scalar_summary channel is being destroyed. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.198.760 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:115] ~MbufDataHandler] Channel ms_scalar_summary begins the destruction process. [INFO] DEVICE(164040,fffedbfff0f0,python):2024-01-10-11:41:09.202.015 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:67] ~ScopeAclTdtDataset] AcltdtDestroyDataset succeed. [INFO] DEVICE(164040,fffedb7fe0f0,python):2024-01-10-11:41:09.202.051 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:67] ~ScopeAclTdtDataset] AcltdtDestroyDataset succeed. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.202.451 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:119] DestoryHandler] The thread of ms_image_summary channel is being destroyed. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.202.475 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:115] ~MbufDataHandler] Channel ms_image_summary begins the destruction process. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.202.763 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:119] DestoryHandler] The thread of ms_tensor_summary channel is being destroyed. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.202.779 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:115] ~MbufDataHandler] Channel ms_tensor_summary begins the destruction process. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.010 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:119] DestoryHandler] The thread of ms_tensor_dump channel is being destroyed. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.029 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:115] ~MbufDataHandler] Channel ms_tensor_dump begins the destruction process. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.312 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:337] CloseTsd] Call aclrtResetDevice, destroy and close tsd successful, ret[0] [INFO] ME(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.341 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:469] ClearDeviceContexts] Release device CPU_0 [INFO] ME(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.359 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:469] ClearDeviceContexts] Release device CPU_1 [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.391 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2260] ClearResPart2] End clear device context. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.403 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2262] ClearResPart2] Start clear AnalysisResultCacheMgr... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.418 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2264] ClearResPart2] End clear AnalysisResultCacheMgr. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.432 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2266] ClearResPart2] Start clear AnalysisContext... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.444 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2268] ClearResPart2] End clear AnalysisContext... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.454 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2270] ClearResPart2] Start clear AnalysisSchedule... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.587 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2272] ClearResPart2] End clear AnalysisSchedule... [INFO] DEBUG(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.617 [mindspore/ccsrc/debug/debugger/debugger.cc:305] Reset] Release Debugger resource. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.658 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2285] ClearResPart3] Start clear ClearObjectCache... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.674 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2287] ClearResPart3] End clear ClearObjectCache... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.688 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2289] ClearResPart3] Start clear Parser... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.701 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2291] ClearResPart3] End clear Parser... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.711 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2293] ClearResPart3] Start ClearTraceStack... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.722 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2295] ClearResPart3] End ClearTraceStack... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.733 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2297] ClearResPart3] Start clear InterpretNodeRecorder... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.744 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2299] ClearResPart3] End clear InterpretNodeRecorder... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.754 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2301] ClearResPart3] Start clear parallel::entire_costgraph... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.770 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2303] ClearResPart3] End clear parallel::entire_costgraph... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.781 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2305] ClearResPart3] Start clear ProtobufLibrary... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.970 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2307] ClearResPart3] End clear ProtobufLibrary... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.986 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2309] ClearResPart3] Start clear python_adapter... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.203.998 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2311] ClearResPart3] End clear python_adapter. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.204.010 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2315] ClearSingleton] Start clear singleton... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.204.136 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2331] ClearSingleton] End clear singleton. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.204.154 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2347] ClearResAtexit] Start unload dynamic lib... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.204.180 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2349] ClearResAtexit] End unload dynamic lib... [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.577.894 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:803] DelOneNetRes] Delete one net resource start [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.584.083 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:825] DelOneNetRes] Delete one net resource end. [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.584.199 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:803] DelOneNetRes] Delete one net resource start [INFO] PIPELINE(164040,ffffb1c2c440,python):2024-01-10-11:41:09.586.550 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:825] DelOneNetRes] Delete one net resource end. [WARNING] PRE_ACT(164040,ffffb1c2c440,python):2024-01-10-11:41:10.044.571 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:793] DumpDynamicMemPoolStateInfo] The dynamic memory pool total allocated mem:2048M, peak used mem:5M, in used mem:0M, total idle mem:71812935M, total eager free mem:0M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [INFO] DEVICE(164040,ffffb1c2c440,python):2024-01-10-11:41:10.044.687 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:136] DeInitialize] DeInitialize Ascend Memory Adapter when it is not initialize [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:41:10.103.488 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:261] DeleteGraphRunner] GraphRunner is not exist [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:41:10.103.559 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:225] DeleteGeSession] Ge Session is not exist [INFO] GE_ADPT(164040,ffffb1c2c440,python):2024-01-10-11:41:10.103.574 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:179] ClearGraph] Remove all graphs in GraphManager [INFO] RUNTIME(164040,python):2024-01-10-11:41:10.827.923 [runtime.cc:1737] 164040 ~Runtime: deconstruct runtime. [INFO] ATRACE(164040,python):2024-01-10-11:41:10.941.223 [atrace_api.c:93](tid:164040) AtraceDestroy start [INFO] ATRACE(164040,python):2024-01-10-11:41:10.941.266 [atrace_api.c:95](tid:164040) AtraceDestroy end [INFO] ATRACE(164043,python):2024-01-10-11:37:02.193.961 [trace_attr.c:105](tid:164043) platform is 1. [INFO] ATRACE(164043,python):2024-01-10-11:37:02.194.167 [trace_recorder.c:114](tid:164043) use root path: /home/jenkins/ascend/atrace [INFO] ATRACE(164043,python):2024-01-10-11:37:02.194.197 [trace_signal.c:133](tid:164043) register signal handler for signo 2 succeed. [INFO] ATRACE(164043,python):2024-01-10-11:37:02.194.209 [trace_signal.c:133](tid:164043) register signal handler for signo 15 succeed. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:02.611.215 [mindspore/core/utils/ms_context.cc:225] set_backend_policy] ms set context backend policy:ge [INFO] RUNTIME(164043,python):2024-01-10-11:37:02.611.332 [runtime.cc:1159] 164043 GetAicoreNumByLevel: workingDev_=0 [INFO] RUNTIME(164043,python):2024-01-10-11:37:02.611.361 [runtime.cc:4719] 164043 GetVisibleDevices: ASCEND_RT_VISIBLE_DEVICES param was not set [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:02.612.060 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:580] SetContextSocVersion] The soc version :Ascend910A [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:02.651.861 [mindspore/ccsrc/pybind_api/ir/log_adapter_py.h:34] PyExceptionInitializer] Set exception handler [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:02.662.572 [mindspore/ccsrc/pipeline/jit/ps/init.cc:179] pybind11_init__c_expression] Start GraphExecutorPy... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:02.663.238 [mindspore/ccsrc/pipeline/jit/ps/init.cc:271] pybind11_init__c_expression] Start ParallelContext... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:02.663.850 [mindspore/ccsrc/pipeline/jit/ps/init.cc:379] pybind11_init__c_expression] Start CostModelContext... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:02.664.282 [mindspore/ccsrc/pipeline/jit/ps/init.cc:481] pybind11_init__c_expression] Start OffloadContext... [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:02.666.120 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:580] SetContextSocVersion] The soc version :Ascend910A [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:04.738.680 [mindspore/core/utils/ms_context.cc:362] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:37:04.738.756 [mindspore/ccsrc/frontend/parallel/costmodel_context.cc:30] GetInstance] Create costmodel_context [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:06.612.295 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:580] SetContextSocVersion] The soc version :Ascend910A [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:06.612.985 [mindspore/core/utils/ms_context.cc:362] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:06.613.116 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:580] SetContextSocVersion] The soc version :Ascend910A [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:06.613.641 [mindspore/core/utils/ms_context.cc:362] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] PS(164043,ffffa1f54440,python):2024-01-10-11:37:06.613.898 [mindspore/ccsrc/ps/ps_context.cc:256] set_ms_role] MS_ROLE of this node is MS_WORKER [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:06.613.966 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:11 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:06.614.156 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:188] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281471256465648 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:06.614.180 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:13 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:06.614.272 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:188] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281471248072944 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:06.614.318 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:15 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:06.614.403 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:188] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281471239680240 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:06.614.423 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:17 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:06.614.507 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:188] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281471231287536 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:06.614.545 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:412] Connect] Can not found link destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:06.614.758 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:18 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:06.614.791 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:456] Connect] Connection 18 source: 127.0.0.1:46014, destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:06.614.814 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:475] Connect] Connected to destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:06.614.828 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:67] Connect] Connected to the tcp server 127.0.0.1:10969 successfully. [INFO] DISTRIBUTED(164043,ffff214170f0,python):2024-01-10-11:37:06.614.825 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:18 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:06.614.841 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:412] Connect] Can not found link destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:06.614.929 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:19 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:06.614.954 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:456] Connect] Connection 19 source: 127.0.0.1:46016, destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:06.614.971 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:475] Connect] Connected to destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:06.614.986 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:67] Connect] Connected to the tcp server 127.0.0.1:10969 successfully. [INFO] DISTRIBUTED(164043,ffff224190f0,python):2024-01-10-11:37:06.614.982 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:19 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:06.615.239 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:209] Register] The compute graph node: 2 has been registered successfully. [WARNING] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:06.615.320 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:195] BuildCluster] Topology build timed out., retry(1/210). [INFO] DISTRIBUTED(164043,ffff13fff0f0,python):2024-01-10-11:37:06.615.385 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:247] Heartbeat] The heartbeat thread is started. [WARNING] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:09.615.397 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:195] BuildCluster] Topology build timed out., retry(2/210). [WARNING] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:12.615.527 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:197] BuildCluster] Cluster is successfully initialized. [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:12.615.587 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:255] PostProcess] Start post processing for computing graph nodes. [WARNING] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:12.615.877 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:261] PostProcess] This node 2 rank id: 2 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:12.615.899 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:268] PostProcess] Client ip address in this cluster of this compute graph node is 127.0.0.1 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:12.615.996 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:279] PostProcess] Port range assigned for this node 2 is 10166 to 11189 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:12.616.040 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:133] node_num] Number of role MS_WORKER is 4 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:12.616.064 [mindspore/ccsrc/distributed/collective/collective_manager.cc:157] Initialize] Start initializing collective communication for backend: Ascend... [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:12.616.116 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:37] MsCollectiveCommLib] Global group name of MindSpore collective communication library is mccl_world_group [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:12.616.129 [mindspore/ccsrc/distributed/collective/collective_manager.cc:412] InitHostCommlib] Start initializing communication library on host side... [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:12.616.153 [mindspore/ccsrc/distributed/collective/collective_manager.cc:432] InitHostCommlib] Communication library on host side is successfully initialized. Global rank id: 2, global rank size: 4 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:12.616.169 [mindspore/ccsrc/distributed/collective/collective_manager.cc:470] AssignLocalRank] Host name for rank 2 is ascend85 [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:12.616.298 [mindspore/ccsrc/distributed/collective/collective_manager.cc:505] AssignLocalRank] The local rank id assigned for this process is 2. device_id of ms_context is set. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:12.627.574 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:624] SetRtDevice] Enter SetRtDevice, current initialize device number:0 [INFO] TDT(164043,python):2024-01-10-11:37:12.630.370 [process_mode_manager.cpp:109][OpenProcess][tid:164043] [ProcessModeManager] enter into open process deviceId[2] rankSize[0] [INFO] TDT(164043,python):2024-01-10-11:37:12.631.154 [process_mode_manager.cpp:379][InitTsdClient][tid:164043] [TsdClient] deviceId[2] begin to init hdc client [INFO] TDT(164043,python):2024-01-10-11:37:12.631.288 [version_verify.cpp:34][SetVersionInfo][tid:164043] VersionVerify: send client version to server [INFO] TDT(164043,python):2024-01-10-11:37:12.631.316 [version_verify.cpp:50][SetVersionInfo][tid:164043] send feature_info:{msg_type:35, features:{check before send aicpu package,}} [INFO] TDT(164043,python):2024-01-10-11:37:12.631.328 [version_verify.cpp:50][SetVersionInfo][tid:164043] send feature_info:{msg_type:37, features:{check before send open qs message,}} [INFO] TDT(164043,python):2024-01-10-11:37:12.631.622 [version_verify.cpp:66][PeerVersionCheck][tid:164043] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(164043,python):2024-01-10-11:37:12.631.638 [version_verify.cpp:87][ParseVersionInfo][tid:164043] VersionVerify: pass client version info success [INFO] TDT(164043,python):2024-01-10-11:37:12.631.647 [hdc_client.cpp:276][CheckHdcConnection][tid:164043] Service[2] create hdc success [INFO] TDT(164043,python):2024-01-10-11:37:12.631.662 [version_verify.cpp:120][SpecialFeatureCheck][tid:164043] VersionVerify: new type[35], supported [INFO] TDT(164043,python):2024-01-10-11:37:12.631.710 [process_mode_manager.cpp:748][GetDeviceCheckCode][tid:164043] [TsdClient][deviceId=2] [sessionId=1] wait package info respond [INFO] TDT(164043,python):2024-01-10-11:37:12.631.819 [process_mode_manager.cpp:379][InitTsdClient][tid:164043] [TsdClient] deviceId[2] begin to init hdc client [INFO] TDT(164043,python):2024-01-10-11:37:12.631.940 [version_verify.cpp:34][SetVersionInfo][tid:164043] VersionVerify: send client version to server [INFO] TDT(164043,python):2024-01-10-11:37:12.631.953 [version_verify.cpp:50][SetVersionInfo][tid:164043] send feature_info:{msg_type:35, features:{check before send aicpu package,}} [INFO] TDT(164043,python):2024-01-10-11:37:12.631.963 [version_verify.cpp:50][SetVersionInfo][tid:164043] send feature_info:{msg_type:37, features:{check before send open qs message,}} [INFO] TDT(164043,python):2024-01-10-11:37:12.632.090 [version_verify.cpp:66][PeerVersionCheck][tid:164043] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(164043,python):2024-01-10-11:37:12.632.102 [version_verify.cpp:87][ParseVersionInfo][tid:164043] VersionVerify: pass client version info success [INFO] TDT(164043,python):2024-01-10-11:37:12.632.110 [hdc_client.cpp:276][CheckHdcConnection][tid:164043] Service[2] create hdc success [INFO] TDT(164043,python):2024-01-10-11:37:12.632.121 [process_mode_manager.cpp:426][ConstructOpenMsg][tid:164043] [TsdClient] tsd get process sign successfully, procpid[164043] signSize[48] [INFO] TDT(164043,python):2024-01-10-11:37:12.632.132 [version_verify.cpp:112][SpecialFeatureCheck][tid:164043] VersionVerify: previous type[6], supported [INFO] TDT(164043,python):2024-01-10-11:37:12.632.153 [process_mode_manager.cpp:126][OpenProcess][tid:164043] [ProcessModeManager] deviceId[2] sessionId[1] rankSize[0], wait sub process start respond [INFO] TDT(164043,python):2024-01-10-11:37:12.815.669 [stub_process_mode_nowin.cpp:63][ProcessQueueForMdc][tid:164043] [TsdClient] it is unnecessary of current mode[0] chiptype[1] to grant queue auth to aicpusd [INFO] TDT(164043,python):2024-01-10-11:37:12.815.717 [stub_process_mode_nowin.cpp:101][OpenInHost][tid:164043] enter into OpenInHost deviceid[2] [INFO] TDT(164043,python):2024-01-10-11:37:12.815.727 [stub_process_mode_nowin.cpp:105][OpenInHost][tid:164043] host cpu not support [INFO] TDT(164043,python):2024-01-10-11:37:12.815.736 [process_mode_manager.cpp:156][OpenProcess][tid:164043] [TsdClient][deviceId=2] [sessionId=1] start hccp and computer process success [INFO] RUNTIME(164043,python):2024-01-10-11:37:12.818.448 [device.cc:340] 164043 Init: isDoubledie:0, topologytype:0 [INFO] RUNTIME(164043,python):2024-01-10-11:37:12.833.569 [npu_driver.cc:5428] 164521 GetDeviceStatus: GetDeviceStatus status=1. [INFO] ATRACE(164043,python):2024-01-10-11:37:12.833.609 [atrace_api.c:28](tid:164043) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:12.833.716 [trace_rb_log.c:84](tid:164043) [RUNTIME_ATRACE_DEV2_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:12.833.732 [atrace_api.c:32](tid:164043) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:12.833.748 [client_manager.cpp:157][SetProfilingCallback][tid:164043] [TsdClient] set profiling callback success [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:12.839.153 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:646] CreateDefaultStream] Create ascend default stream, stream id: 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:12.840.358 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:652] CreateDefaultStream] Create ascend communication stream, stream id: 1 [INFO] DEBUG(164043,ffffa1f54440,python):2024-01-10-11:37:12.840.400 [mindspore/ccsrc/debug/debugger/debugger.cc:101] Debugger] Debugger got device_target: Ascend [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:12.840.602 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:116] Initialize] Device HBM Size:32768M, Device free HBM Size:32735M, Reserved HBM size for Other Components(HCCL/rts/etc.):2057M, Recommend Reserved HBM size for Other Components:2045M, User define MindSpore HBM Size:0G, MindSpore Used HBM Size:30678M. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:12.994.105 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:301] MallocFromRts] Call rtMalloc to allocate device memory Success, size: 32168214528 bytes, address start: 0x124100000000 end: 0x12487d600000 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:12.994.151 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:129] Initialize] Ascend Memory Adapter initialize success, Memory Statistics: Device HBM memory size: 32768M MindSpore Used memory size: 30678M MindSpore memory base address: 0x124100000000 Total Static Memory size: 0M Total Dynamic memory size: 0M Dynamic memory size of this graph: 0M [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:12.995.407 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:463] SetDisableReuseMemoryFlag] DISABLE_REUSE_MEMORY is not set in ENV. Now set to default value 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:12.995.435 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:500] SetHcclOptions] No hccl mode. If use hccl, make sure [RANK_TABLE_FILE,RANK_ID,DEVICE_ID] all be set in ENV. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:12.995.462 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:366] GetGeOptions] JOB_ID is not set in ENV. Now set to default value 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:12.995.478 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:384] GetGeOptions] Set proto lib path failed! [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:12.995.490 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:304] SetAscendConfig] GE topo sorting mode is: [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:12.995.503 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:316] SetAscendConfig] Set GE topo mode to memory-priority. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:12.995.515 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:321] SetAscendConfig] Set staticMemoryPolicy to default mode. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:12.995.527 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:329] SetAscendConfig] The default value of jit_compile is set to 2. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:12.995.538 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:285] SetAscendHF32Config] The default value of allow_matmul_hf32 and allow_conv_hf32 are set by CANN. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:12.995.549 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:294] SetAscendHF32Config] allow_matmul_hf32: , allow_conv_hf32: [TRACE] GE(164043,python):2024-01-10-11:37:12.995.586 [status:INIT] [ge_api.cc:144]164043 GEInitializeImpl:GEInitialize start [INFO] PROFILING(164043,python):2024-01-10-11:37:13.211.074 [msprofiler_impl.cpp:156] >>> (tid:164043) ProfNotifySetDevice called, is open: 1, devId: 2 [INFO] PROFILING(164043,python):2024-01-10-11:37:13.211.229 [platform.cpp:38] >>> (tid:164043) Profiling platform version: 1.0. [INFO] PROFILING(164043,python):2024-01-10-11:37:13.211.245 [ai_drv_dev_api.cpp:384] >>> (tid:164043) Succeeded to DrvGetApiVersion version: 0x72313 [TRACE] GE(164043,python):2024-01-10-11:37:13.262.982 [status:RUNNING] [ge_api.cc:211]164043 GEInitializeImpl:Initializing environment [INFO] GE(164043,python):2024-01-10-11:37:13.263.059 [gelib.cc:98][EVENT]164043 Initialize:[GEPERFTRACE] GE Init Start [INFO] GE(164043,python):2024-01-10-11:37:13.263.320 [gelib.cc:307][EVENT]164043 SystemInitialize:Online infer init GELib success, device id :2 [INFO] DVPP(164043,python):2024-01-10-11:37:13.640.627 [dvpp_engine.cc:41][ENGINE][Initialize:41][tid:164043]dvpp engine do not support [INFO] TUNE(164043,python):2024-01-10-11:37:13.645.080 [cann_kb_pyfunc_mgr.cpp:72][CANNKB][Tid:164043]"CannKbPyfuncMgr: Enter PyObjectInit, reference_ is 0!" [INFO] TUNE(164043,python):2024-01-10-11:37:13.645.116 [handle_manager.cpp:115][CANNKB][Tid:164043]"Start to run init functions to load dynamic python lib!" [INFO] TUNE(164043,python):2024-01-10-11:37:13.645.174 [handle_manager.cpp:407][CANNKB][Tid:164043]"Init functions of loading dynamic python lib end!" [INFO] TUNE(164043,python):2024-01-10-11:37:13.645.185 [cann_kb_pyfunc_mgr.cpp:24][CANNKB][Tid:164043]"CANN_KB_Py has already been initialized." [INFO] TUNE(164043,python):2024-01-10-11:37:13.645.255 [cann_kb_pyfunc_mgr.cpp:117][CANNKB][Tid:164043]"CannKbPyfuncMgr: Run PyObjectInit successfully!" [INFO] HCCL(164043,python):2024-01-10-11:37:25.332.496 [plugin_manager.cc:42][164043]hcom running normal mode. [INFO] DVPP(164043,python):2024-01-10-11:37:25.333.118 [dvpp_engine.cc:92][ENGINE][GetOpsKernelInfoStores:92][tid:164043]dvpp ops kernel info store do not support [INFO] DVPP(164043,python):2024-01-10-11:37:25.333.270 [dvpp_engine.cc:69][ENGINE][GetGraphOptimizerObjs:69][tid:164043]dvpp graph optimizer do not support [INFO] DVPP(164043,python):2024-01-10-11:37:26.021.598 [dvpp_ops_kernel_builder.cc:48][ENGINE][Initialize:48][tid:164043]dvpp ops kernel builder do not support [INFO] GE(164043,python):2024-01-10-11:37:26.032.465 [gelib.cc:169][EVENT]164043 Initialize:[GEPERFTRACE] The time cost of GELib::Initialize is [12769356] micro second. [TRACE] GE(164043,python):2024-01-10-11:37:26.120.708 [status:STOP] [ge_api.cc:255]164043 GEInitializeImpl:GEInitialize finished [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:26.120.886 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_res_manager.cc:168] GeSetContextOptions] Set GE atomic clean policy to 1. [TRACE] GE(164043,python):2024-01-10-11:37:26.120.954 [status:INIT] [ge_api.cc:398]164043 Session:Start to construct session. [TRACE] GE(164043,python):2024-01-10-11:37:26.120.972 [status:RUNNING] [ge_api.cc:408]164043 Session:Creating session [INFO] GE(164043,python):2024-01-10-11:37:26.121.416 [graph_var_manager.cc:1445][EVENT]164043 SetMemoryMallocSize:Total memory size is 34359738368 [INFO] GE(164043,python):2024-01-10-11:37:26.121.433 [graph_var_manager.cc:1424][EVENT]164043 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] PROFILING(164043,python):2024-01-10-11:37:26.121.807 [msprofiler_impl.cpp:156] >>> (tid:164043) ProfNotifySetDevice called, is open: 1, devId: 2 [TRACE] GE(164043,python):2024-01-10-11:37:26.122.645 [status:RUNNING] [ge_api.cc:411]164043 Session:Session id is 0 [TRACE] GE(164043,python):2024-01-10-11:37:26.122.667 [status:STOP] [ge_api.cc:420]164043 Session:Session Constructor finished [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:26.122.699 [mindspore/ccsrc/transform/graph_ir/graph_runner.cc:53] NewSession] Create new GE session success! [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:26.122.722 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:210] SetGeSession] Add a new Ge Session success [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:26.122.752 [mindspore/ccsrc/transform/graph_ir/graph_runner.cc:65] GraphRunner] ME run in ONE_DEVICE strategy mode [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:26.122.809 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:246] SetGraphRunner] Add a new GraphRunner success [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:26.122.844 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:238] InitGe] Create session and graphrunner successful. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:26.122.858 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:242] InitGe] Init ge successful, ge reference = 1. [INFO] PROFILING(164043,python):2024-01-10-11:37:26.133.051 [platform.cpp:38] >>> (tid:164043) Profiling platform version: 1.0. [INFO] PROFILING(164043,python):2024-01-10-11:37:26.133.083 [ai_drv_dev_api.cpp:384] >>> (tid:164043) Succeeded to DrvGetApiVersion version: 0x72313 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:26.133.253 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:193] Initialize] Call aclInit successfully. [TRACE] GE(164043,python):2024-01-10-11:37:26.133.321 [status:INIT] [ge_api.cc:144]164043 GEInitializeImpl:GEInitialize start [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:26.134.642 [mindspore/ccsrc/distributed/collective/collective_manager.cc:455] InitDeviceCommLib] Start initializing communication library on device side... [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:26.134.708 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:291] OpenTsd] Device id = 2, rank size = 4. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:26.143.302 [mindspore/ccsrc/plugin/device/ascend/hal/device/tensorprint_utils.cc:405] CreateChannel] For Print ops, select MBUF channel. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:26.143.336 [mindspore/ccsrc/plugin/device/ascend/hal/device/tensorprint_utils.cc:420] CreateTensorPrintThread] Success to create acl channel handle, tsd reference = 1. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:26.143.984 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:103] MbufDataHandler] Channel ms_tensor_dump begins the construction process. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:26.144.477 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:103] MbufDataHandler] Channel ms_tensor_summary begins the construction process. [INFO] DEVICE(164043,fffed9d860f0,python):2024-01-10-11:37:26.144.524 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:161] HandleData] Channel ms_tensor_dump starts executing HandleData. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:26.144.891 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:103] MbufDataHandler] Channel ms_image_summary begins the construction process. [INFO] DEVICE(164043,fffed95850f0,python):2024-01-10-11:37:26.144.941 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:161] HandleData] Channel ms_tensor_summary starts executing HandleData. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:26.145.317 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:103] MbufDataHandler] Channel ms_scalar_summary begins the construction process. [INFO] DEVICE(164043,fffed8d840f0,python):2024-01-10-11:37:26.145.365 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:161] HandleData] Channel ms_image_summary starts executing HandleData. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:26.145.651 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:103] MbufDataHandler] Channel ms_histogram_summary begins the construction process. [INFO] DEVICE(164043,fffec3fff0f0,python):2024-01-10-11:37:26.145.713 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:161] HandleData] Channel ms_scalar_summary starts executing HandleData. [INFO] HCCL_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:26.145.990 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:202] InitHccl] Start init hccl adapter. [INFO] DEVICE(164043,fffec37fe0f0,python):2024-01-10-11:37:26.146.051 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:161] HandleData] Channel ms_histogram_summary starts executing HandleData. [INFO] HCCL_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:26.146.130 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:341] InitKernelInfoStore] Start init hccl kernel info store. [INFO] HCCL_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:26.146.188 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:358] InitKernelInfoStore] Get builder ops_kernel_info_hccl [INFO] HCCL(164043,python):2024-01-10-11:37:26.146.218 [plugin_manager.cc:42][164043]hcom running normal mode. [INFO] HCCL_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:26.146.290 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:380] InitKernelInfoStore] Init hccl kernel info store success. [INFO] HCCL_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:26.146.308 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:528] InitHcclExec] Start init hccl exec. [INFO] HCCL(164043,python):2024-01-10-11:37:26.149.151 [hcom_executor.cc:32][164043][Initialize][HcomExecutor]Hcom Excutor Initialize end. ret[0] [INFO] HCCL_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:26.149.197 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:534] InitHcclExec] Hcom DynamicKernel Initialize success [INFO] HCCL_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:26.149.221 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:540] InitHcclExec] InitHcclExec success [INFO] HCCL_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:26.149.234 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:220] InitHccl] Init hccl adapter success. [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:26.149.250 [mindspore/ccsrc/distributed/collective/collective_manager.cc:458] InitDeviceCommLib] Communication library on device side is successfully initialized. [WARNING] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:26.149.269 [mindspore/ccsrc/distributed/collective/collective_manager.cc:220] CreateCommunicationGroup] Start to create communication group: hccl_world_group [const vector]{0, 1, 2, 3} [WARNING] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:26.149.624 [mindspore/ccsrc/distributed/collective/collective_manager.cc:278] CreateCommunicationGroup] Begin initialize communication group on the device side: hccl_world_group [INFO] HCCL(164043,python):2024-01-10-11:37:26.149.810 [externalinput.cc:310][166217]environmental variable HCCL_CONNECT_TIMEOUT is set, timeOut[600] [INFO] HCCL(164043,python):2024-01-10-11:37:26.149.855 [externalinput.cc:598][166217]environmental variable HCCL_IF_IP is not set [INFO] HCCL(164043,python):2024-01-10-11:37:26.149.869 [externalinput.cc:655][166217]environmental variable HCCL_SOCKET_IFNAME is not set, default[EmptyString] [INFO] HCCL(164043,python):2024-01-10-11:37:26.149.884 [externalinput.cc:582][166217]environmental variable HCCL_IF_BASE_PORT is not set [INFO] HCCL(164043,python):2024-01-10-11:37:26.149.970 [externalinput.cc:282][166217]environmental variable HCCL_HIGH_PERF_ENABLE is not set [INFO] HCCL(164043,python):2024-01-10-11:37:26.149.999 [op_base.cc:405][166217]Entry-HcclCommInitRootInfo:ranks[4], rank[2], rootinfo: host ip[8.92.9.85] port[60000] nicDeploy[1] identifier[8.92.9.85%enp189s0f0_60000_0_1704857845691656], deviceLogicId[2] [INFO] HCCP(164043,python):2024-01-10-11:37:26.150.327 [ra_host.c:1882]tid:166217,ra_get_ifnum(1882) : Input parameters: phy_id[0], nic_position:[0] [INFO] HCCL(164043,python):2024-01-10-11:37:26.152.459 [adapter_hccp.cc:821][166217][Get][HostIf]hrtGetIfNum success. ifAddrNum[7]. [INFO] HCCP(164043,python):2024-01-10-11:37:26.152.482 [ra_host.c:1930]tid:166217,ra_get_ifaddrs(1930) : Input parameters: phy_id[0], nic_position:[0], interface num[7] [INFO] HCCL(164043,python):2024-01-10-11:37:26.153.636 [sal.cc:383][166217]nic class[normal]: find nic[8.92.9.85%enp189s0f0] success. [INFO] HCCP(164043,python):2024-01-10-11:37:26.153.718 [ra_host.c:1722]tid:166217,ra_socket_set_white_list_status(1722) : Input parameters: enable[0] [INFO] HCCP(164043,python):2024-01-10-11:37:26.153.749 [ra_host.c:293]tid:166217,ra_init(293) : Input parameters: phy_id[2], nic_position:[0] [INFO] HCCP(164043,python):2024-01-10-11:37:26.154.059 [rs_ssl.c:1104]tid:166217,rs_ssl_init(1104) : TLS SWITCH (0) [INFO] HCCP(164043,python):2024-01-10-11:37:26.154.260 [rs_epoll.c:470]tid:166218,rs_epoll_handle(470) : pthread[epoll_pthread] is alive! [INFO] HCCP(164043,python):2024-01-10-11:37:26.154.259 [rs_epoll.c:595]tid:166219,rs_connect_handle(595) : pthread[connect_pthread] is alive! [INFO] HCCP(164043,python):2024-01-10-11:37:26.154.285 [rs.c:403]tid:166217,rs_init(403) : rs init success, chip_id[2] [INFO] TDT(164043,python):2024-01-10-11:37:26.154.334 [process_mode_manager.cpp:109][OpenProcess][tid:166217] [ProcessModeManager] enter into open process deviceId[2] rankSize[2] [INFO] TDT(164043,python):2024-01-10-11:37:26.154.683 [process_mode_manager.cpp:705][GetDeviceCheckCode][tid:166217] [ProcessModeManager][deviceId=2] aicpu package already exist in device [INFO] TDT(164043,python):2024-01-10-11:37:26.154.726 [process_mode_manager.cpp:426][ConstructOpenMsg][tid:166217] [TsdClient] tsd get process sign successfully, procpid[164043] signSize[48] [INFO] TDT(164043,python):2024-01-10-11:37:26.154.783 [process_mode_manager.cpp:126][OpenProcess][tid:166217] [ProcessModeManager] deviceId[2] sessionId[1] rankSize[2], wait sub process start respond [INFO] TDT(164043,python):2024-01-10-11:37:26.307.377 [stub_process_mode_nowin.cpp:63][ProcessQueueForMdc][tid:166217] [TsdClient] it is unnecessary of current mode[0] chiptype[1] to grant queue auth to aicpusd [INFO] TDT(164043,python):2024-01-10-11:37:26.307.401 [stub_process_mode_nowin.cpp:101][OpenInHost][tid:166217] enter into OpenInHost deviceid[2] [INFO] TDT(164043,python):2024-01-10-11:37:26.307.411 [stub_process_mode_nowin.cpp:105][OpenInHost][tid:166217] host cpu not support [INFO] TDT(164043,python):2024-01-10-11:37:26.307.420 [process_mode_manager.cpp:156][OpenProcess][tid:166217] [TsdClient][deviceId=2] [sessionId=1] start hccp and computer process success [INFO] HCCP(164043,python):2024-01-10-11:37:26.307.432 [ra_host.c:293]tid:166217,ra_init(293) : Input parameters: phy_id[2], nic_position:[1] [INFO] HCCP(164043,python):2024-01-10-11:37:26.307.459 [ra_hdc.c:1465]tid:166217,ra_hdc_init(1465) : hdc init start! logic id is 2, phy id is 2 [INFO] HCCP(164043,python):2024-01-10-11:37:26.307.815 [ra_hdc.c:1500]tid:166217,ra_hdc_init(1500) : hdc init OK! phy_id[2] [INFO] HCCP(164043,python):2024-01-10-11:37:26.310.318 [ra_host.c:454]tid:166217,ra_socket_init_v1(454) : socket init:mode=0 phy_id=2 family=2 ip=8.92.9.85 [INFO] HCCL(164043,python):2024-01-10-11:37:26.310.351 [adapter_hccp.cc:988][166217][Get][DeviceIP]hrtGetIfNum success. ifAddrNum[2]. [INFO] HCCP(164043,python):2024-01-10-11:37:26.310.360 [ra_host.c:1930]tid:166217,ra_get_ifaddrs(1930) : Input parameters: phy_id[2], nic_position:[1], interface num[2] [INFO] HCCL(164043,python):2024-01-10-11:37:26.314.863 [adapter_hccp.cc:1018][166217]hrtGetIfAddress: idx[0] ifname[eth2] ip[192.168.102.101%eth2] [INFO] HCCL(164043,python):2024-01-10-11:37:26.314.877 [topoinfo_detect.cc:472][166217]select AF_INET family as device socket family. [INFO] HCCP(164043,python):2024-01-10-11:37:26.314.924 [ra_host.c:825]tid:166217,ra_socket_batch_connect(825) : Input parameters: [0]th, phy_id[2], local_ip[8.92.9.85], remote_ip[8.92.9.85], tag:[topo_detect_default_tag_60000] [INFO] HCCP(164043,python):2024-01-10-11:37:26.693.312 [ra_host.c:863]tid:166217,ra_socket_batch_close(863) : Input parameters: [0]th, phy_id[2], local_ip[8.92.9.85] [INFO] HCCP(164043,python):2024-01-10-11:37:26.733.830 [ra_host.c:525]tid:166217,ra_socket_deinit(525) : Input parameters: phy_id[2] family[2] local_ip[8.92.9.85] [INFO] HCCP(164043,python):2024-01-10-11:37:26.733.860 [rs.c:1257]tid:166217,rs_socket_deinit(1257) : socket deinit success, phy_id:2, local_ip:8.92.9.85 [INFO] HCCP(164043,python):2024-01-10-11:37:26.733.875 [ra_host.c:349]tid:166217,ra_deinit(349) : Input parameters: phy_id[2], nic_position:[0] [INFO] HCCP(164043,python):2024-01-10-11:37:26.753.985 [rs.c:1460]tid:166217,rs_deinit(1460) : rs_deinit chip_id[2] ok [INFO] HCCP(164043,python):2024-01-10-11:37:26.754.004 [ra_host.c:349]tid:166217,ra_deinit(349) : Input parameters: phy_id[2], nic_position:[1] [INFO] HCCP(164043,python):2024-01-10-11:37:26.754.015 [ra_hdc.c:1535]tid:166217,ra_hdc_deinit(1535) : hdc deinit start! phy_id[2] [INFO] HCCP(164043,python):2024-01-10-11:37:26.754.155 [ra_hdc.c:1570]tid:166217,ra_hdc_deinit(1570) : hdc deinit OK! phy_id[2] [INFO] ATRACE(164043,python):2024-01-10-11:37:26.754.351 [atrace_api.c:28](tid:166217) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:26.754.403 [trace_rb_log.c:84](tid:166217) [HCCL_166217_2] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:26.754.414 [atrace_api.c:32](tid:166217) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:26.754.813 [process_mode_manager.cpp:109][OpenProcess][tid:166217] [ProcessModeManager] enter into open process deviceId[2] rankSize[2] [INFO] HCCP(164043,python):2024-01-10-11:37:26.754.833 [ra_host.c:293]tid:166217,ra_init(293) : Input parameters: phy_id[2], nic_position:[1] [INFO] HCCP(164043,python):2024-01-10-11:37:26.754.910 [ra_hdc.c:1465]tid:166217,ra_hdc_init(1465) : hdc init start! logic id is 2, phy id is 2 [INFO] HCCP(164043,python):2024-01-10-11:37:26.755.125 [ra_hdc.c:1500]tid:166217,ra_hdc_init(1500) : hdc init OK! phy_id[2] [INFO] HCCP(164043,python):2024-01-10-11:37:26.757.529 [ra_host.c:389]tid:166217,ra_socket_init(389) : socket init:mode=1 phy_id=2 family=2 ip=2.0.0.0 [INFO] HCCP(164043,python):2024-01-10-11:37:26.758.204 [ra_host.c:903]tid:166217,ra_socket_listen_start(903) : Input parameters: [0]th, phy_id[2], local_ip[2.0.0.0] [INFO] HCCL(164043,python):2024-01-10-11:37:26.761.537 [hccl_impl.cc:430][166217]hccl algorithm: [Module(aiserver)] there are 4 device in level0, using fullmesh algo [TRACE] HCCL(164043,python):2024-01-10-11:37:26.766.143 [status:init] [op_base.cc:481][166217]HcclCommInitRootInfo success,take time [616345]us, rankNum[4], rank[2],rootInfo identifier[8.92.9.85%enp189s0f0_60000_0_1704857845691656], server[8.92.9.85%enp189s0f0], device[2] [WARNING] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:26.766.199 [mindspore/ccsrc/distributed/collective/collective_manager.cc:287] CreateCommunicationGroup] End initialize communication group on the device side: hccl_world_group [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:37:26.766.226 [mindspore/ccsrc/distributed/collective/collective_manager.cc:182] Initialize] End initializing collective communication for backend: Ascend [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:26.766.266 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:285] RecordInitStatus] Status record: system init. [WARNING] ME(164043:281473398948928,MainProcess):2024-01-10-11:37:28.478.118 [mindspore/parallel/_utils.py:259] You are suggested to use mindspore.context.set_auto_parallel_context(parameter_broadcast=True) or mindspore.common.set_seed() to share parameters among multi-devices. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.235.224 [mindspore/ccsrc/minddata/dataset/util/task_manager.cc:161] DoServiceStart] Starting Task Manager. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.235.930 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at train-labels-idx1-ubyte. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.235.962 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at train-images-idx3-ubyte. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.236.099 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.237.891 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.238.013 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.238.035 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.238.051 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.238.082 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.238.148 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.238.174 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.238.188 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.238.209 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.238.227 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.238.259 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.238.272 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.238.292 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.238.319 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.238.599 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at train-labels-idx1-ubyte. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.238.624 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at train-images-idx3-ubyte. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.889.287 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.889.407 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.935.769 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.935.906 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.935.929 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.935.945 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.935.976 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.936.041 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.936.068 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.936.084 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.936.105 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.936.124 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.936.156 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.936.170 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.936.214 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.936.243 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.936.569 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at train-labels-idx1-ubyte. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:29.936.596 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at train-images-idx3-ubyte. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.532.771 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.532.901 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.579.328 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.579.462 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.579.485 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.579.502 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.579.532 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.579.600 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.579.625 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.579.639 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.579.683 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.579.703 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.579.736 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.579.750 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.579.770 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.579.799 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.580.087 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] ME(164043:281473398948928,MainProcess):2024-01-10-11:37:30.629.862 [mindspore/dataset/engine/datasets.py:4269] queue_name is newly generated. value is 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.632.522 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.632.645 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.632.668 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.632.685 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.632.734 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.632.801 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.632.828 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.632.844 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.632.866 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.632.884 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.632.916 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.632.930 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.632.950 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.632.979 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.633.280 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:30.633.464 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1729] InitExecDatasetVm] Start InitDataSet Entry [INFO] DEBUG(164043,ffffa1f54440,python):2024-01-10-11:37:30.633.700 [mindspore/ccsrc/common/debug/env_config_parser.cc:152] ParseFromFile] The 'env_config_path' in 'mindspore.context.set_context(env_config_path={path})' is empty. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:37:30.633.734 [mindspore/ccsrc/backend/graph_compiler/transform.cc:573] CreateBackend] CreateBackend is: ge [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:30.633.751 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:30.633.764 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEBUG(164043,ffffa1f54440,python):2024-01-10-11:37:30.633.886 [mindspore/ccsrc/debug/debugger/debugger.cc:129] Init] Debugger got device_id: 2 [INFO] DEBUG(164043,ffffa1f54440,python):2024-01-10-11:37:30.633.910 [mindspore/ccsrc/debug/debugger/debugger.cc:131] Init] Debugger got device_target: Ascend [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:30.724.591 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:433] Initialize] The actor thread number: 5, the kernel thread number: 25 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:30.724.825 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:30.724.850 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:30.724.864 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1252] SetRunMode] Run graph mode with kernel by kernel by configuration. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:37:30.725.335 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:546] CompileGraphs] Status record: start compile function graph: _anonymous__1 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.725.475 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unify_mindir_pm_0_erase_invalid_micro_depend in 4.24 us [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:37:30.726.190 [mindspore/ccsrc/utils/anfalgo.cc:1736] IsNodeOutputDynamicShape] Invalid base shape, node: Default/Return-op0 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:30.726.258 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:30.726.275 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:30.726.313 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:30.726.326 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:30.726.347 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:30.726.361 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:37:30.726.391 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:727] CompileGraph] Compile graph: _anonymous__1, Split segments size: 2 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:37:30.726.427 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:762] CompileGraphFromSegment] Compile normal segment, the first node: @_anonymous__1:CNode_2{[0]: ValueNode InitDataSetQueue} [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:30.726.510 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:421] CompileGraph] Status record: start compile graph. [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:30.726.565 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:1940] ConstructKernelGraph] Create graph: 0 [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:30.726.723 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:2697] ConstructOutput] Output:@_anonymous__1:CNode_2{[0]: ValueNode InitDataSetQueue} [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.727.119 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:193] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:0 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.727.190 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 8.12 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.727.316 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 78.07 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.727.428 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:154] CommonUnifyMindIR] start common unify mindir opt graph:0 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.727.476 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.727.560 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 78.59 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.727.578 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.727.613 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 30.7 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.727.629 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.727.644 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 13.92 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.727.657 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.727.701 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 40.48 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.727.815 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:79] BackendCommonOptimization] Status record: start common optimization. graph id: 0 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.727.858 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: add_dynamic_shape_attr [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.727.907 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_0_add_dynamic_shape_attr in 43.9 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.727.923 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.727.975 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_1_convert_dynamic_broadcast_to in 47.61 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.728.021 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_2_convert_const_input_to_attr in 27.53 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.728.056 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_3_custom_op_const_input_to_attr in 14.92 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.728.086 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 12 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.728.173 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 65.21 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.728.194 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 0.78 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.728.264 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_7_flatten_concat_fission in 37.09 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.728.311 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 25.3 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.728.354 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_9_broadcast_to_fusion in 22.69 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.728.507 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 129.95 us [WARNING] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.728.612 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:92] BackendCommonOptimization] [PROF]backend_common_optimization costs 798 usec. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.728.630 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:99] BackendCommonOptimization] Status record: end common optimization. graph id: 0 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.728.791 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_0_renorm_split in 37.05 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.728.812 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.728.883 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_1_reduce_axis_update in 65.53 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.728.902 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim HistogramFixedWidth not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.728.918 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_2_histogram_fixed_width_fusion in 15 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.728.940 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim ClipByNorm not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.728.955 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_3_clip_by_norm_fission in 21.39 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.728.979 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.728.994 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_4_tensor_arry_add_flow_cond1 in 23.3 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.007 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayGather not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.021 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_5_tensor_arry_add_flow_cond2 in 12 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.033 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.047 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_6_ge_tensor_arry_cast_index in 12.11 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.065 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArray not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.097 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_7_tensor_array_prepare in 34.2 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.110 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.150 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_8_space_to_batch_nd_attr_update in 35.96 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.165 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.192 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_9_batch_to_space_nd_attr_update in 24.44 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.206 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: avg_pool_grad_unify_mindir [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.249 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_10_avg_pool_grad_unify_mindir in 39.15 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.302 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_11_adam_weight_decay_unify_mindir in 33.08 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.352 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_12_cdist_fission in 28.16 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.394 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_13_cdist_grad_fission in 21.11 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.477 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_14_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 59.75 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.530 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_15_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 33.31 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.570 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_16_sparse_softmax_cross_entropy_with_logits_unify_mindir in 20.26 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.606 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_17_dropout_unify_mindir1 in 17.95 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.622 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.667 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_18_dropoutgrad_unify_mindir in 41.34 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.724 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.746 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_19_neighbor_exchange_unify_mindir in 60.1 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.760 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.774 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_20_neighbor_exchange_v2_unify_mindir in 12.74 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.796 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.810 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_21_neighbor_exchange_v2_grad_unify_mindir in 12.27 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.829 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.843 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_22_all_to_all_unify_mindir in 18.56 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.857 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim QuantDTypeCast not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.871 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_23_quant_dtype_cst_adjust in 13.48 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.885 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim FSEDecode not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.899 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_24_fse_decode_adjust in 13.72 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.917 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.932 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_25_bn_split in 18.06 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.729.944 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.007 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_26_bn_grad_unify_mindir in 59.01 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.024 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.039 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_27_bn_grad_split in 14.02 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.052 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.065 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_28_batchnorm_to_bninfer in 11.45 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.078 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.091 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_29_batchnormgrad_to_bninfergrad in 11.48 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.103 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.116 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_30_batch_norm_grad_infer_fission in 11.31 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.166 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_31_ascend_mindir_op_adapter in 32.91 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.194 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_32_DropoutGenMask in 1.7 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.237 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_33_lamb_fission_ge in 25.31 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.293 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_34_print_insert_placeholder_for_tensor_name in 35.74 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.342 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_35_getnext_for_ge in 28.59 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.387 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_36_sync_bn_split in 25.31 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.427 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_37_sync_bn_grad_split in 19.71 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.475 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_38_adaptive_max_pool2d_ge_fusion in 29.05 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.512 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_39_avg_pool_grad_for_ge in 17.45 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.534 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:260] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.645 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_40_FlashAttentionFusionV1 in 112.46 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.665 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:374] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.730.775 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_41_FlashAttentionFusionV1 in 107.68 us [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:30.731.111 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:30.731.137 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:30.731.151 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:132] GetRunMode] RunMode::kKernelMode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.731.296 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 106.91 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.731.480 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_0_seed_adapter in 54.94 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.731.505 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: env_op_attr_update [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.731.552 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_1_env_op_attr_update in 43.6 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.731.607 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_2_process call inline in 24.75 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.731.640 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_3_expander_fallback in 12.6 us [WARNING] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.731.719 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:139] GEBackendOptimizeACL] [PROF]ascend_backend_optimize_acl costs 308 usec. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:30.731.753 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] InitDataSetQueue is not defined in opdef. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.731.974 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 11.06 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.732.051 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 47.68 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.732.114 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 37.4 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.732.188 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 51.06 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.732.228 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_4_insert_type_transform_op in 18.17 us [WARNING] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.732.309 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:179] GEBackendOptimizeACLAfterKernelSelect] [PROF]ascend_backend_optimize_acl_after_kernel_select costs 409 usec. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.732.362 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_0_DropoutGenMask in 0.52 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.732.415 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_1_cse in 31.48 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.732.470 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_redundant_op in 35.01 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.732.497 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_3_eliminate_maketuple_getitem in 8.91 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.732.515 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_4_MergeTransData in 0.89 us [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:30.732.633 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive InitDataSetQueue [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:30.732.737 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive InitDataSetQueue [WARNING] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:30.732.756 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:262] CreateKernel] [PROF]create_kernel costs 131 usec. [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:30.732.809 [mindspore/ccsrc/backend/common/session/session_basic.cc:1140] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 0 start [INFO] DEBUG(164043,ffffa1f54440,python):2024-01-10-11:37:30.732.835 [mindspore/ccsrc/debug/summary/summary.cc:33] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 0 start [INFO] DEBUG(164043,ffffa1f54440,python):2024-01-10-11:37:30.732.851 [mindspore/ccsrc/debug/summary/summary.cc:54] RecurseSetSummaryNodesForAllGraphs] The total summary nodes is: 0 for graph: 0 [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:30.732.913 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:891] PrintGraphExecuteOrder] Graph 0 execution order: [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:30.732.957 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[0], node name[Default/InitDataSetQueue-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_0:CNode_2{[0]: ValueNode InitDataSetQueue}] [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.003 [mindspore/ccsrc/backend/common/somas/somas.cc:143] Assign] Start Somas Assign for graph 0 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.031 [mindspore/ccsrc/backend/common/somas/somas.cc:151] Assign] Somas Configure success, configuration info: Device Name: Ascend Run by execution order: 1 Enable debug log: 0 Debug log path: . [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.043 [mindspore/ccsrc/backend/common/somas/somas.cc:154] Assign] Start Initialize SOMAS Model [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.097 [mindspore/ccsrc/backend/common/somas/somas.cc:1065] GraphOutputProcess] Set 0 graph output tensors' aligned size to 0. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.133 [mindspore/ccsrc/backend/common/somas/somas.cc:1135] RefNodeProcess] Special Tensor total size: RefNode: input 0 output 0 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.154 [mindspore/ccsrc/backend/common/somas/somas.cc:551] InitSomasModel] No Tensor from graph 0 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.167 [mindspore/ccsrc/backend/common/somas/somas.cc:157] Assign] End Initialize SOMAS Model [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.177 [mindspore/ccsrc/backend/common/somas/somas.cc:160] Assign] No Somas Tensor in graph 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.188 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:310] DoSomas] Somas allocate success for graph 0 somas size: 0 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.213 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:801] CreateDeviceAddress] Status record: start create device address. graph id: 0 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.276 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:808] CreateDeviceAddress] Status record: end create device address. graph id: 0 [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.310 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1129] CacheGraphOutputToFrontNodeWithIndex] Get graph backend output nodes. [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.330 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1137] CacheGraphOutputToFrontNodeWithIndex] Get graph front output nodes. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.348 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:507] CompileGraph] Status record: end compile graph. graph id: 0 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.429 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:805] CompileGraphFromSegment] Compile cut segment, the cut node: @_anonymous__1:CNode_3{[0]: ValueNode Return, [1]: CNode_2} [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.495 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:521] Transform] Graph(kernel_graph_0) transforms actor begin, strategy:pipeline_with_execution_order [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.595 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1232] BuildLoopCountActor] Create loop count actor: kernel_graph_0_LoopCountActor [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.614 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1265] BuildOutputActor] Create output actor: kernel_graph_0_OutputActor [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.632 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1285] BuildDataPrepareActor] Create data prepare actor: kernel_graph_0_DataPrepareActor [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.662 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1518] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_0 start. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.679 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1520] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_0 end. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.801 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 1_actor_set_kernel_graph_0_memory_actor_insert in 1.26 us [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.825 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 2_actor_set_kernel_graph_0_invalid_data_arrow_elimination in 0.989996 us [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.858 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 3_actor_set_kernel_graph_0_multi_actor_fusion in 15.55 us [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.873 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 4_actor_set_kernel_graph_0_batch_data_arrow_fusion in 1.52001 us [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.890 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:554] Transform] Graph(kernel_graph_0) transforms actor end. [WARNING] VM(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.940 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:612] CompileGraphs] [PROF]compile_func_graph costs 8579 usec. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.962 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:615] CompileGraphs] Status record: end compile function graph: _anonymous__1, produce actor: kernel_graph_0 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:37:30.733.983 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_0 [INFO] GE(164043,python):2024-01-10-11:37:30.734.430 [scalable_config.cc:55][EVENT]167074 ScalableConfig:device total max size: 34359738368, page_mem_size_total_thresold: 32641751449, uncacheable_size_threshold: 17179869184 [INFO] GE(164043,python):2024-01-10-11:37:30.813.900 [graph_var_manager.cc:1424][EVENT]167074 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:30.814.006 [graph_manager.cc:1248][EVENT]167074 PreRun:PreRun start: graph node size 1, session id 1, graph id 0, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:30.814.930 [atrace_api.c:28](tid:167074) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:30.815.008 [trace_rb_log.c:84](tid:167074) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:30.815.023 [atrace_api.c:32](tid:167074) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:30.815.055 [client_manager.cpp:157][SetProfilingCallback][tid:167074] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:30.815.994 [parallel_partitioner.cc:165][EVENT]167074 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [24] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.816.056 [parallel_partitioner.cc:178][EVENT]167074 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.816.106 [graph_prepare.cc:1378][EVENT]167074 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.816.769 [graph_manager.cc:1050][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [682] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.816.796 [graph_manager.cc:1052][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.816.868 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [1] [INFO] GE(164043,python):2024-01-10-11:37:30.816.896 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.816.971 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [64] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.816.984 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.817.078 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.817.092 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.817.103 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.817.203 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [3] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.817.227 [graph_manager.cc:1054][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [419] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.824.759 [graph_manager.cc:1055][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [7517] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.825.348 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:30.825.377 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.825.389 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.825.398 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [115] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.825.407 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.825.416 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:30.825.424 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.825.433 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.825.451 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [9] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.826.505 [graph_manager.cc:1056][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [1708] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.826.564 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.826.583 [graph_prepare.cc:1982][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [43] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.826.735 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:30.826.752 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.826.762 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.826.771 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [48] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.826.779 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [4] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.826.788 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:30.826.796 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [0] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.826.805 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [0] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.826.813 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.826.846 [graph_prepare.cc:1983][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [250] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.826.867 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.826.878 [graph_prepare.cc:1984][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.826.891 [graph_prepare.cc:1985][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.826.910 [graph_prepare.cc:1986][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.826.922 [graph_prepare.cc:1987][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.826.936 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.826.948 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.826.961 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.827.039 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.827.051 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.827.061 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.827.069 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.827.078 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.827.086 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.827.094 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [0] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.827.103 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.827.112 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.827.120 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.827.128 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.827.137 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.827.145 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.827.153 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.827.162 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.827.170 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.827.190 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.827.202 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.827.227 [graph_prepare.cc:1988][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [296] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.827.240 [graph_manager.cc:1065][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [701] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.839.777 [graph_manager.cc:1077][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12517] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.839.826 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.839.852 [graph_manager.cc:1080][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.842.513 [graph_manager.cc:1081][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [2637] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.842.557 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.842.571 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.842.582 [graph_manager.cc:1082][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.842.610 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.842.622 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.842.636 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.842.674 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [28] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.842.688 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.842.701 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.842.715 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.842.746 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.842.771 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.842.787 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.842.804 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.842.820 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.842.832 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.842.841 [graph_manager.cc:2700][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [236] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.842.942 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.842.955 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.842.964 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.842.973 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.842.990 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.842.999 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CastRemovePass is [26] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.843.008 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.843.016 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.843.024 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.843.033 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.843.041 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [4] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.843.049 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [0] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.843.057 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.843.066 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.843.074 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.843.084 [graph_manager.cc:2741][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [223] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.093 [graph_manager.cc:2752][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.115 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.126 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.141 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.156 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.167 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.178 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.204 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.217 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.230 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.240 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.260 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.271 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.282 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.295 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.305 [graph_manager.cc:2810][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [195] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.328 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.843.341 [graph_manager.cc:2821][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.367 [graph_manager.cc:1087][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [769] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.497 [graph_manager.cc:1088][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [118] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.534 [graph_manager.cc:1089][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.551 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.564 [graph_manager.cc:1097][EVENT]167074 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:30.843.584 [graph_manager.cc:3325][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.697 [engine_place.cc:144][EVENT]167074 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.712 [engine_place.cc:144][EVENT]167074 Run:The time cost of aicpu_ascend_kernel::CheckSupported is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.778 [graph_manager.cc:3351][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [180] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.793 [graph_manager.cc:3364][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.858 [engine_partitioner.cc:1139][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.874 [engine_partitioner.cc:1142][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.843.988 [engine_partitioner.cc:1148][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [105] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.844.014 [engine_partitioner.cc:1155][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.844.067 [engine_partitioner.cc:1164][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.844.099 [graph_manager.cc:3405][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [292] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.844.117 [graph_manager.cc:3412][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.316 [graph_manager.cc:3422][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [2185] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.352 [graph_manager.cc:3428][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.460 [graph_manager.cc:3467][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [89] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.478 [graph_manager.cc:3377][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [2673] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.494 [graph_manager.cc:1106][EVENT]167074 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [2915] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.506 [graph_manager.cc:1115][EVENT]167074 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:30.846.527 [graph_manager.cc:1130][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.559 [graph_manager.cc:1131][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.585 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.601 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.611 [graph_manager.cc:2837][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.656 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.846.667 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.846.676 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondRemovePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.846.685 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.846.693 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.846.701 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:30.846.711 [graph_manager.cc:2864][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [84] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.731 [graph_manager.cc:2872][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.749 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.763 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.778 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.791 [compile_nodes_pass.cc:88][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.801 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.813 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.848 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [24] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.873 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.885 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.898 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.910 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.919 [graph_manager.cc:2927][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [172] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.939 [graph_manager.cc:2937][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.968 [graph_manager.cc:2943][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.846.987 [graph_manager.cc:2950][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.856.662 [graph_manager.cc:2958][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.856.709 [graph_manager.cc:1132][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [10136] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.856.821 [graph_manager.cc:1135][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [95] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.856.869 [graph_manager.cc:2975][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [27] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.856.954 [graph_manager.cc:2981][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [61] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.856.970 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.856.980 [graph_manager.cc:2986][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.856.990 [graph_manager.cc:1136][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [149] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.857.067 [graph_manager.cc:3555][EVENT]167074 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [51] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.857.129 [engine_partitioner.cc:1139][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.857.144 [engine_partitioner.cc:1142][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.857.216 [engine_partitioner.cc:1148][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [62] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.857.238 [engine_partitioner.cc:1155][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.857.271 [engine_partitioner.cc:1164][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.857.291 [graph_builder.cc:865][EVENT]167074 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [188] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.857.366 [graph_builder.cc:288][EVENT]167074 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [50] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.857.466 [graph_builder.cc:293][EVENT]167074 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [87] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.857.727 [model_builder.cc:1133][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [142] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.857.975 [block_mem_assigner.cc:4069][EVENT]167126 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164043,python):2024-01-10-11:37:30.857.976 [block_mem_assigner.cc:4069][EVENT]167127 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164043,python):2024-01-10-11:37:30.858.240 [model_builder.cc:1144][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [487] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.858.269 [model_builder.cc:1152][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.858.284 [model_builder.cc:1157][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.858.417 [model_builder.cc:1167][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [122] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.858.436 [model_builder.cc:1174][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.858.466 [model_builder.cc:1180][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.858.509 [model_builder.cc:1184][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.858.529 [graph_builder.cc:304][EVENT]167074 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [1038] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:30.858.760 [logger.cc:1071] 167074 ModelBindStream: model_id=576, stream_id=1857, flag=0. [INFO] GE(164043,python):2024-01-10-11:37:30.858.838 [task_generator.cc:804][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.858.902 [task_generator.cc:805][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [48] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.859.427 [task_generator.cc:814][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [511] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.859.441 [task_generator.cc:954][EVENT]167074 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [610] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.859.505 [task_generator.cc:967][EVENT]167074 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [36] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:30.859.524 [logger.cc:1084] 167074 ModelUnbindStream: model_id=576, stream_id=1857, [INFO] GE(164043,python):2024-01-10-11:37:30.859.580 [graph_builder.cc:310][EVENT]167074 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [1038] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.859.681 [graph_manager.cc:1152][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2674] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.859.697 [graph_manager.cc:1164][EVENT]167074 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:30.859.735 [graph_manager.cc:1271][EVENT]167074 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [43856] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.859.746 [graph_manager.cc:1272][EVENT]167074 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:37:30.860.057 [atrace_api.c:93](tid:167074) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:37:30.860.080 [atrace_api.c:95](tid:167074) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:37:30.860.677 [model_introduction.cc:236][EVENT]167074 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(164043,python):2024-01-10-11:37:30.860.698 [model_introduction.cc:294][EVENT]167074 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(164043,python):2024-01-10-11:37:30.860.711 [model_introduction.cc:366][EVENT]167074 Data:model io_info size:0 [INFO] GE(164043,python):2024-01-10-11:37:30.862.903 [graph_converter.cc:838][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [672] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.862.966 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.863.176 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [194] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.863.234 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [39] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.863.247 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [53] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.863.280 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.863.304 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.863.323 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.863.357 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.863.400 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [31] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.863.410 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [42] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.863.428 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.863.447 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.863.460 [graph_converter.cc:849][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [516] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.863.574 [graph_converter.cc:853][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [105] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.863.972 [graph_converter.cc:857][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [384] micro second. [INFO] GE(164043,python):2024-01-10-11:37:30.864.056 [graph_converter.cc:862][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [61] micro second. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:30.865.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_0_LoopCountActor) running, loop count: 1, current count: 1, total running count: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:30.865.685 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_0 execution count: 1, execution time: 131.63 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:37:30.865.828 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:30.865.959 [mindspore/ccsrc/runtime/device/kernel_runtime_manager.cc:35] ClearGraphResource] Clear device Ascend_2 graph 0 runtime resource [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.869.130 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:198] Compile] Input plan: +-Transfer,send_epoch_end:false,total_batch:2340) | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.869.249 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:216] Compile] Plan before optimization: +-Top | +-Transfer,send_epoch_end:false,total_batch:2340) | | +-Repeat(count:1) | | | +-Batch(batch_size:32 drop_remainder:true) | | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.869.282 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:60] PrePass] Running pre pass loops. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.869.300 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.869.332 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.869.414 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.869.445 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.869.462 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.869.486 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.869.501 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/cache_transform_pass.cc:182] RunOnTree] Pre pass: Cache transform pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.869.526 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/cache_transform_pass.cc:199] RunOnTree] Pre pass: Cache transform pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.869.538 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:91] PrePass] Pre pass offload complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.869.556 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:116] PostPass] Running post pass loops. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.869.588 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:135] PostPass] Post passes complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.869.626 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:230] Compile] Plan after optimization: +-Top | +-Transfer,send_epoch_end:false,total_batch:2340) | | +-EpochCtrl(epoch:5) | | | +-Batch(batch_size:32 drop_remainder:true) | | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | | +-MnistDataset [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:30.870.308 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_data_queue.cc:227] AscendTdtQueue] Select MBUF channel, the capacity of data queue is: 128 [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:37:30.870.376 [mindspore/ccsrc/minddata/dataset/engine/datasetops/epoch_ctrl_op.cc:25] EpochCtrlOp] Welcome to Epoch Ctrl Op. [INFO] MD(164043,fffdca7fc0f0,python):2024-01-10-11:37:30.873.186 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at train-labels-idx1-ubyte. [INFO] MD(164043,fffdca7fc0f0,python):2024-01-10-11:37:30.873.244 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at train-images-idx3-ubyte. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:30.877.558 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:456] SendDataToAscend] Device queue, sending data to Ascend. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.229.598 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:493] GenerateArgumentsKey] Generate a new compile key for new args, key: 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.229.676 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:501] GenerateArgumentsKey] New cached args: [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.230.705 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:978] CompileInner] Start compiling, phase: train.1704857851074634240.281471696508592.0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.230.742 [mindspore/ccsrc/pipeline/jit/ps/event_message_print.cc:37] PrintEventMessage] Start compiling '_DataWrapper.construct' and it will take a while. Please wait... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.238.842 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1659] VmPipeline] This worker is initialized. No need to add worker action. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:37:31.238.922 [mindspore/ccsrc/backend/graph_compiler/transform.cc:573] CreateBackend] CreateBackend is: ge [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.238.946 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.238.960 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEBUG(164043,ffffa1f54440,python):2024-01-10-11:37:31.239.250 [mindspore/ccsrc/debug/debugger/debugger.cc:129] Init] Debugger got device_id: 2 [INFO] DEBUG(164043,ffffa1f54440,python):2024-01-10-11:37:31.239.270 [mindspore/ccsrc/debug/debugger/debugger.cc:131] Init] Debugger got device_target: Ascend [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.239.294 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1311] Run] Pipeline run [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.239.314 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start parse action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.250.632 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end parse action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.250.692 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start symbol_resolve action. [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.270.445 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] _no_sens_impl_4 update var `grads` with node @_no_sens_impl_4:grads{[0]: CNode_5, [1]: grads} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.270.691 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_no_sens_impl_4] Added global python symbol: {F : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.271.183 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] _no_sens_impl_4 update var `loss` with node @_no_sens_impl_4:loss{[0]: CNode_6, [1]: loss, [2]: CNode_7} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.288.937 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] mindspore_nn_optim_momentum_Momentum_construct_8 update var `gradients` with node @mindspore_nn_optim_momentum_Momentum_construct_8:gradients{[0]: CNode_9, [1]: param_gradients} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.289.263 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] mindspore_nn_optim_momentum_Momentum_construct_8 update var `gradients` with node @mindspore_nn_optim_momentum_Momentum_construct_8:gradients{[0]: CNode_10, [1]: gradients} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.289.542 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] mindspore_nn_optim_momentum_Momentum_construct_8 update var `gradients` with node @mindspore_nn_optim_momentum_Momentum_construct_8:gradients{[0]: CNode_11, [1]: gradients} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.289.837 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] mindspore_nn_optim_momentum_Momentum_construct_8 update var `gradients` with node @mindspore_nn_optim_momentum_Momentum_construct_8:gradients{[0]: CNode_12, [1]: gradients} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.290.565 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_13{[0]: CNode_14, [1]: CNode_15, [2]: CNode_16}, block: 0x1bcead30/mindspore_nn_optim_momentum_Momentum_construct_8, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/nn/optim/momentum.py:211/ self.assignadd(self.global_step, self.global_step_increase_tensor)/ [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.291.156 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_optim_momentum_Momentum_construct_8] Added global python symbol: {F : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.291.397 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_optim_momentum_Momentum_construct_8] Added global python symbol: {_momentum_opt : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.292.046 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_17{[0]: ValueNode Depend, [1]: CNode_18, [2]: CNode_19}, state: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_13{[0]: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_14{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.optim.momentum..', [2]: ValueNode assignadd}, [1]: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_15{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.optim.momentum..', [2]: ValueNode global_step}, [2]: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_16{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.optim.momentum..', [2]: ValueNode global_step_increase_tensor}} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.306.716 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20] Added global python symbol: {F : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.306.968 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20] Added global python symbol: {_get_datatype : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.307.487 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20] Added global python symbol: {_cast_datatype : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.307.677 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20] Added global python symbol: {mstype : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.307.955 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20 update var `grads` with node @mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20:grads{[0]: CNode_21, [1]: CNode_22, [2]: param_grads} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.308.618 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20] Added global python symbol: {reduce_opt : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.319.049 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23] Added global python symbol: {_check_is_tensor : Prim[_check_is_tensor]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.319.555 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_24{[0]: CNode_25, [1]: ValueNode logits, [2]: param_logits, [3]: CNode_26}, block: 0x1bcead30/mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/nn/loss/loss.py:777/ _check_is_tensor('logits', logits, self.cls_name)/ [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.320.094 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_27{[0]: CNode_25, [1]: ValueNode labels, [2]: param_labels, [3]: CNode_28}, block: 0x1bcead30/mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/nn/loss/loss.py:778/ _check_is_tensor('labels', labels, self.cls_name)/ [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.320.741 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_29{[0]: ValueNode Depend, [1]: CNode_30, [2]: CNode_31}, state: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_32{[0]: ValueNode MakeTuple, [1]: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_24{[0]: CNode_25, [1]: ValueNode logits, [2]: param_logits, [3]: CNode_26}, [2]: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_27{[0]: CNode_25, [1]: ValueNode labels, [2]: param_labels, [3]: CNode_28}} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.330.557 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_34, [1]: param_x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.330.853 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_35, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.331.147 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_36, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.331.422 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_37, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.331.695 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_38, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.331.972 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_39, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.332.250 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_40, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.332.517 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_41, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.332.976 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_42, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.333.321 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_43, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.333.622 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_44, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.333.939 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_45, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.347.238 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_46] Added global python symbol: {len : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.347.421 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.347.793 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.347.973 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.348.475 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_46:CNode_48{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.348.623 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_47:x{[0]: CNode_49, [1]: param_фx, [2]: CNode_48} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.349.107 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_46:CNode_50{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.349.832 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_47] Added global python symbol: {len : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.349.906 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_46:CNode_51{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.350.022 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_46:x_shape{[0]: CNode_52, [1]: param_x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.350.170 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.350.477 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.350.770 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_47] Added global python symbol: {F : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.350.868 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_46] Added global python symbol: {F : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.350.921 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_46:CNode_53{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.351.307 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.358.518 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_54] Added global python symbol: {len : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.358.691 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.359.032 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.359.202 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.359.694 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_54:CNode_56{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.359.864 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_55:x{[0]: CNode_57, [1]: param_фx, [2]: CNode_56} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.360.337 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_54:CNode_58{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.360.811 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_55] Added global python symbol: {len : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.360.878 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_54:CNode_59{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.360.991 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_54:x_shape{[0]: CNode_60, [1]: param_x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.361.138 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.361.453 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.361.745 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_55] Added global python symbol: {F : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.361.843 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_54] Added global python symbol: {F : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.361.901 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_54:CNode_61{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.362.286 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.366.127 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_62] Added global python symbol: {len : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.366.300 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.366.646 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.366.819 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.367.304 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_62:CNode_64{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.367.450 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_63:x{[0]: CNode_65, [1]: param_фx, [2]: CNode_64} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.367.907 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_62:CNode_66{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.368.378 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_63] Added global python symbol: {len : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.368.445 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_62:CNode_67{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.368.559 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_62:x_shape{[0]: CNode_68, [1]: param_x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.368.709 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.369.009 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.369.288 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_63] Added global python symbol: {F : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.369.383 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_62] Added global python symbol: {F : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.369.439 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_62:CNode_69{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.369.813 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.373.798 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Flatten_construct_70] Added global python symbol: {F : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.387.855 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1261] ParseNameConstant] The NameConstant is bool:False [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.388.195 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:3 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.388.466 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.388.605 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1261] ParseNameConstant] The NameConstant is bool:True [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.389.334 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_pooling_MaxPool2d_construct_71] Added global python symbol: {isinstance : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.389.441 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_pooling_MaxPool2d_construct_72] Added global python symbol: {isinstance : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.389.499 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_pooling_MaxPool2d_construct_71 update var `isinstance` with node @mindspore_nn_layer_pooling_MaxPool2d_construct_72:CNode_73{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.pooling', [2]: ValueNode isinstance} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.389.672 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_pooling_MaxPool2d_construct_71] Added global python symbol: {tuple : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.389.847 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_pooling_MaxPool2d_construct_72] Added global python symbol: {tuple : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.389.898 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_pooling_MaxPool2d_construct_71 update var `tuple` with node @mindspore_nn_layer_pooling_MaxPool2d_construct_72:CNode_74{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.pooling', [2]: ValueNode tuple} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.390.280 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.390.402 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.390.615 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.390.728 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.391.018 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.432.935 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.433.113 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.433.831 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @canonicalize_axis_75:CNode_76{[0]: ValueNode check_axis_valid_77, [1]: param_axis, [2]: ndim}, block: 0x284fd340/canonicalize_axis_75, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1606/ check_axis_valid(axis, ndim)/ [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.434.023 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.434.344 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @canonicalize_axis_75:CNode_78{[0]: ValueNode Depend, [1]: CNode_79, [2]: CNode_80}, state: @canonicalize_axis_75:CNode_76{[0]: ValueNode check_axis_valid_77, [1]: @canonicalize_axis_75:param_axis, [2]: @canonicalize_axis_75:ndim{[0]: CNode_81}} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.434.639 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {isinstance : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.434.814 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {Tensor : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.435.401 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {int : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.435.877 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {bool : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.436.590 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {check_flatten_order_const : Prim[check_flatten_order]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.437.208 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @2↓flatten_83:CNode_84{[0]: CNode_85, [1]: param_order}, block: 0x28524df0/2↓flatten_83, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1615/ check_flatten_order_const(order)/ [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.437.662 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {rank_ : Prim[Rank]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.438.109 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.438.181 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.438.420 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {reshape_ : Prim[Reshape]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.438.618 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.438.940 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {ops : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.439.159 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.439.697 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {transpose_ : Prim[Transpose]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.440.127 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_86] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.440.258 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.440.328 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `shape_` with node @flatten_82:CNode_87{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode shape_} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.440.731 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_86] Added global python symbol: {rank_ : Prim[Rank]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.440.806 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `rank_` with node @flatten_82:CNode_88{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode rank_} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.441.129 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `start_dim` with node @flatten_82:param_start_dim [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.441.289 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.441.443 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `end_dim` with node @flatten_82:param_end_dim [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.441.565 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.441.906 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.441.964 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.442.216 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_86] Added global python symbol: {reshape_ : Prim[Reshape]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.442.289 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `reshape_` with node @flatten_82:CNode_89{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode reshape_} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.442.513 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.442.854 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_86] Added global python symbol: {flatten_ : Prim[Flatten]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.442.964 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {flatten_ : Prim[Flatten]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.443.038 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `flatten_` with node @flatten_82:CNode_90{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode flatten_} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.443.393 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `canonicalize_axis` with node ValueNode canonicalize_axis_75 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.443.871 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `check_dim_valid` with node ValueNode check_dim_valid_91 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.444.440 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @4↓flatten_92:CNode_93{[0]: ValueNode check_dim_valid_91, [1]: start_dim, [2]: end_dim}, block: 0x285510b0/4↓flatten_92, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1636/ check_dim_valid(start_dim, end_dim)/ [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.444.694 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.444.752 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.445.033 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.445.642 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.446.269 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.446.874 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.447.351 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @2↓flatten_83:CNode_94{[0]: ValueNode Depend, [1]: CNode_95, [2]: CNode_96}, state: @2↓flatten_83:CNode_84{[0]: @flatten_82:CNode_85{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode check_flatten_order_const}, [1]: @flatten_82:param_order} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.447.469 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @4↓flatten_92:CNode_97{[0]: ValueNode Depend, [1]: CNode_98, [2]: CNode_99}, state: @4↓flatten_92:CNode_93{[0]: ValueNode check_dim_valid_91, [1]: @4↓flatten_92:idx{[0]: ValueNode canonicalize_axis_75, [1]: param_start_dim, [2]: x_rank}, [2]: @4↓flatten_92:end_dim{[0]: ValueNode canonicalize_axis_75, [1]: param_end_dim, [2]: x_rank}} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.447.583 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.447.690 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.448.921 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:174] CheckFuncReturn] Function must has 'return' statement, but missing in function 'flatten' at /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1557. FuncGraph: ↓check_dim_valid_100. We will add a 'return None' statement automatically. [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.449.040 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:174] CheckFuncReturn] Function must has 'return' statement, but missing in function 'flatten' at /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1557. FuncGraph: ↓check_axis_valid_101. We will add a 'return None' statement automatically. [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.493.509 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [shape_102] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.502.354 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end symbol_resolve action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.502.411 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start graph_reusing action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.502.429 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.basic.DenseDense[True, None]_ID [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.502.442 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.conv.Conv2dConv2dodict_values([6, 16, 5, 1, 'valid', 0, False, ])_ID [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.502.453 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.conv.Conv2dConv2dodict_values([1, 6, 5, 1, 'valid', 0, False, ])_ID [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.502.468 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end graph_reusing action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.502.487 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start meta_unpack_prepare action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.503.643 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end meta_unpack_prepare action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.503.690 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pre_cconv action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.503.708 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pre_cconv action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.503.728 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start abstract_specialize action. [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.526.112 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_106{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.526.189 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.527.190 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_108{[0]: CNode_109}, new_node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_110{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.527.249 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_108{[0]: CNode_109}, new node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_108{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.530.231 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_equal_scalar_112] Added global python symbol: {F : } [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.530.624 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 4, Shape: NoShape), AbstractScalar(Type: Int64, Value: 3, Shape: NoShape)}, g: _equal_scalar_112 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.531.369 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_113:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_113:CNode_115{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.531.436 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_113:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_113:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.534.878 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_117{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.534.946 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.535.278 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_118{[0]: CNode_119}, new_node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_120{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.535.332 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_118{[0]: CNode_119}, new node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_118{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.535.979 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_121:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_121:CNode_122{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.536.048 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_121:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_121:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.543.361 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_logical_not_scala_124] Added global python symbol: {auto_generate : } [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.543.828 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Bool, Value: true, Shape: NoShape)}, g: _logical_not_scala_124 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.549.540 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bool_not_125] Added global python symbol: {_get_cache_prim : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.549.750 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bool_not_125] Added global python symbol: {BoolNot : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.564.060 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_126] Added global python symbol: {str : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.564.516 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↻_get_cache_prim_for_pynative_127] Added global python symbol: {str : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.564.795 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↻_get_cache_prim_for_pynative_127 update var `str` with node @↵_get_cache_prim_for_pynative_128:param_фstr [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.565.028 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_126] Added global python symbol: {tuple : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.565.235 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] _get_cache_prim_for_pynative_129 update var `key` with node @_get_cache_prim_for_pynative_129:key{[0]: CNode_130, [1]: key, [2]: CNode_131} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.566.137 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_132] Added global python symbol: {str : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.566.788 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_132] Added global python symbol: {_PRIM_CACHE : {}} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.566.888 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_126] Added global python symbol: {_PRIM_CACHE : {}} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.567.148 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_132] Added global python symbol: {Primitive : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.567.241 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_126] Added global python symbol: {Primitive : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.567.972 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @✓↓_get_cache_prim_for_pynative_133:CNode_134{[0]: ValueNode MetaFuncGraph-unpack_call.135, [1]: CNode_136, [2]: param_фargs, [3]: param_фkwargs}, block: 0x2b267bc0/✓↓_get_cache_prim_for_pynative_133, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/_primitive_cache.py:84/ prim.__init__(*args, **kwargs)/ [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.568.602 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 2↓_get_cache_prim_for_pynative_137 update var `key` with node @↓_get_cache_prim_for_pynative_138:key{[0]: param_фstr, [1]: param_фkey} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.568.773 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @↻_get_cache_prim_for_pynative_139:CNode_140{[0]: ValueNode Depend, [1]: CNode_141, [2]: CNode_142}, state: @↻_get_cache_prim_for_pynative_139:CNode_143{[0]: ValueNode MetaFuncGraph-add.144, [1]: @↵_get_cache_prim_for_pynative_132:param_@CNode_143, [2]: ValueNode 1} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.568.902 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @✓↓_get_cache_prim_for_pynative_133:CNode_145{[0]: ValueNode Depend, [1]: CNode_146, [2]: CNode_147}, state: @✓↓_get_cache_prim_for_pynative_133:CNode_134{[0]: ValueNode MetaFuncGraph-unpack_call.135, [1]: @✓↓_get_cache_prim_for_pynative_133:CNode_136{[0]: ValueNode getattr, [1]: prim, [2]: ValueNode __init__}, [2]: @↵_get_cache_prim_for_pynative_132:param_фargs, [3]: @↵_get_cache_prim_for_pynative_132:param_фkwargs} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.570.293 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_148:CNode_149{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.570.360 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_148:CNode_150{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.570.405 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_148:CNode_151{[0]: ValueNode MakeTuple} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.570.975 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:502] SendDataToAscend] Begin to send data to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.571.037 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:1182] PrintBeginInfoWhenFirstBatch] Loading dataset and begin to push first batch into device ... [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.571.088 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BoolNot. node: @bool_not_125:CNode_152{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x}, new_node: @bool_not_125:CNode_153{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.571.146 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BoolNot. node: @bool_not_125:CNode_152{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x}, new node: @bool_not_125:CNode_152{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.572.599 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:1192] PrintEndInfoWhenFirstBatch] Loading dataset and push first batch into device successful. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.572.628 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.573.133 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.575.176 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 3 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.575.624 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 4 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.576.526 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 5 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.577.393 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_equal_string_154] Added global python symbol: {F : } [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.577.652 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 6 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.577.786 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: String, Value: C, Shape: NoShape), AbstractScalar(Type: String, Value: F, Shape: NoShape)}, g: _equal_string_154 [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.578.806 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 7 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.579.364 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_155:CNode_156{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_155:CNode_157{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.579.434 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_155:CNode_156{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_155:CNode_156{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.579.774 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 8 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.580.745 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 9 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.581.799 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 10 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.582.098 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_neg_scalar_159] Added global python symbol: {F : } [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.582.440 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 1, Shape: NoShape)}, g: _neg_scalar_159 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.583.127 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarUsub. node: @_neg_scalar_160:CNode_161{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x}, new_node: @_neg_scalar_160:CNode_162{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.583.189 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarUsub. node: @_neg_scalar_160:CNode_161{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x}, new node: @_neg_scalar_160:CNode_161{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.583.297 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 11 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.583.886 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_163:CNode_164{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_163:CNode_165{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.583.955 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_163:CNode_164{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_163:CNode_164{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.584.418 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @3↓flatten_166:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput}, new_node: @3↓flatten_166:CNode_167{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.584.487 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @3↓flatten_166:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput}, new node: @3↓flatten_166:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.584.507 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 12 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.585.360 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 13 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.586.528 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 14 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.587.607 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 15 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.588.663 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 16 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.588.766 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_169] Added global python symbol: {F : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.589.496 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_169] Added global python symbol: {InSequence : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.589.909 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_169] Added global python symbol: {const_utils : } [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.590.143 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 17 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.590.426 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 4, Shape: NoShape), AbstractTuple{ element[0]: AbstractScalar(Type: Int64, Value: 0, Shape: NoShape), element[1]: AbstractScalar(Type: Int64, Value: 1, Shape: NoShape), sequence_nodes: {@✓3↓flatten_170:CNode_171{[0]: ValueNode MakeTuple, [1]: ValueNode 0, [2]: ValueNode 1}, elements_use_flags: {ptr: 0x2b2cead0, value: [const vector]{0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: _number_in_tuple_169 [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.590.810 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 18 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.592.439 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 19 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.593.608 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 20 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.594.004 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Flatten. node: @↓✓3↓flatten_172:CNode_173{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput}, new_node: @↓✓3↓flatten_172:CNode_174{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.594.082 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Flatten. node: @↓✓3↓flatten_172:CNode_173{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput}, new node: @↓✓3↓flatten_172:CNode_173{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.594.445 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_175:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_175:CNode_176{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.594.503 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_175:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_175:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.594.547 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 21 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.595.583 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 22 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.596.663 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 23 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.597.488 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_not_equal_scalar_178] Added global python symbol: {F : } [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.597.827 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 24 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.597.918 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 2, Shape: NoShape), AbstractScalar(Type: Int64, Value: 2, Shape: NoShape)}, g: _not_equal_scalar_178 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.598.742 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_179:CNode_180{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_179:CNode_181{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.598.813 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_179:CNode_180{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_179:CNode_180{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.598.881 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 25 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.599.933 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 26 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.600.878 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_183:CNode_184{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_183:CNode_185{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.600.950 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_183:CNode_184{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_183:CNode_184{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.601.671 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 27 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.602.201 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_186:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_186:CNode_187{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias, [3]: ValueNode 0} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.602.280 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_186:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_186:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias, [3]: ValueNode 0} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.602.609 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_188{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.602.666 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.602.858 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_189:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_189:CNode_190{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.602.911 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_189:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_189:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.602.970 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 28 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.603.792 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_191:CNode_192{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_191:CNode_193{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.603.846 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 29 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.603.869 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_191:CNode_192{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_191:CNode_192{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.605.046 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 30 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.605.991 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_194:CNode_195{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_194:CNode_196{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.606.034 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 31 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.606.071 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_194:CNode_195{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_194:CNode_195{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.607.125 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 32 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.607.325 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_197:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_197:CNode_198{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias, [3]: ValueNode 0} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.607.401 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_197:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_197:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias, [3]: ValueNode 0} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.607.674 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_199{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.607.728 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.607.915 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_200:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_200:CNode_201{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.607.983 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_200:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_200:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.608.210 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 33 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.608.862 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_202:CNode_203{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_202:CNode_204{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.608.934 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_202:CNode_203{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_202:CNode_203{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.609.311 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 34 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.610.831 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_205:CNode_206{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_205:CNode_207{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.610.905 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_205:CNode_206{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_205:CNode_206{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.610.945 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 35 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.612.144 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_208:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_208:CNode_209{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias, [3]: ValueNode 0} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.612.210 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 36 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.612.229 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_208:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_208:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias, [3]: ValueNode 0} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.613.045 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 37 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.614.128 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 38 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.615.214 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 39 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.615.336 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny)}, g: hyper_map_212 [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.616.225 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 40 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.617.328 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 41 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.617.916 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_ones_like_tensor_213] Added global python symbol: {P : } [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.618.295 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny)}, g: _ones_like_tensor_213 [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.618.498 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 42 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.619.441 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: OnesLike. node: @_ones_like_tensor_214:grads{[0]: ValueNode PrimFunc_OnesLike, [1]: param_x}, new_node: @_ones_like_tensor_214:CNode_215{[0]: ValueNode PrimFunc_OnesLike, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.619.509 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_OnesLike. node: @_ones_like_tensor_214:grads{[0]: ValueNode PrimFunc_OnesLike, [1]: param_x}, new node: @_ones_like_tensor_214:grads{[0]: ValueNode PrimFunc_OnesLike, [1]: param_x} [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.619.918 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: env_get(AbstractScalar(Type: Object:EnvType, Value: ValueAny, Shape: NoShape), )}, AbstractTuple{ element[0]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[1]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[2]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[3]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[4]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[5]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[6]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[7]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), sequence_nodes: {@_no_sens_impl_216:CNode_217{[0]: ValueNode MakeTuple, [1]: param_conv1.weight, [2]: param_conv2.weight, [3]: param_fc1.weight, [4]: param_fc1.bias, [5]: param_fc2.weight, [6]: param_fc2.bias, [7]: param_fc3.weight, [8]: param_fc3.bias}, elements_use_flags: {ptr: 0x2b3c39e0, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: hyper_map_218 [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.620.036 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 43 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.620.303 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: env_get(AbstractScalar(Type: Object:EnvType, Value: ValueAny, Shape: NoShape), )}, AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny)}, g: hyper_map_219 [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.621.483 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 44 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.622.436 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 45 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.623.182 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensor_env_get_221] Added global python symbol: {environ_get : Prim[EnvironGet]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.623.400 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensor_env_get_221] Added global python symbol: {ref_to_embed : Prim[RefToEmbed]} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.623.468 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 46 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.623.669 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensor_env_get_221] Added global python symbol: {tensor_zeros_like : Prim[ZerosLike]} [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.623.988 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Object:EnvType, Value: ValueAny, Shape: NoShape), AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny)}, g: _tensor_env_get_221 [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.624.542 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 47 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.625.021 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_222:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_222:CNode_223{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.625.090 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_222:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_222:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.625.646 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 48 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.626.227 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_224:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_224:CNode_225{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.626.299 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_224:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_224:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.626.785 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 49 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.627.398 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_226:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_226:CNode_227{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.627.469 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_226:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_226:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.627.848 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 50 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.628.568 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_228:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_228:CNode_229{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.628.649 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_228:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_228:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.629.304 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 51 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.629.853 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_230:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_230:CNode_231{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.629.924 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_230:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_230:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.630.876 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 52 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.631.021 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_232:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_232:CNode_233{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.631.092 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_232:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_232:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.631.484 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 53 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.632.192 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_234:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_234:CNode_235{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.632.261 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_234:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_234:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.632.562 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 54 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.633.370 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_236:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_236:CNode_237{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.633.456 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_236:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_236:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.633.629 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 55 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.634.379 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: S-signature()}, AbstractTuple{ element[0]: AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[1]: AbstractTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[2]: AbstractTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[3]: AbstractTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[4]: AbstractTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[5]: AbstractTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[6]: AbstractTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[7]: AbstractTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), sequence_nodes: {@hyper_map_239:grads{[0]: ValueNode MakeTuple, [1]: grads, [2]: grads, [3]: grads, [4]: grads, [5]: grads, [6]: grads, [7]: grads, [8]: grads}, elements_use_flags: {ptr: 0x2b4594c0, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: map_240 [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.634.739 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 56 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.635.820 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 57 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.636.936 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 58 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.637.328 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensors_get_datatype_242] Added global python symbol: {F : } [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.637.696 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny)}, g: _tensors_get_datatype_242 [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.638.508 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 59 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.639.573 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 60 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.640.486 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 61 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.641.547 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 62 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.642.643 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 63 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.643.697 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 64 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.644.880 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 65 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.645.814 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 66 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.646.954 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensors_cast_datatype_244] Added global python symbol: {F : } [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.647.342 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 67 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.647.331 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractType(Value: Float32), AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny)}, g: _tensors_cast_datatype_244 [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.648.581 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 68 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.649.388 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 69 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.650.584 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 70 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.651.658 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 71 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.652.759 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 72 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.653.906 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 73 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.654.957 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 74 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.655.323 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: S-signature(AbstractTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x252d6f20, value: Tensor(shape=[], dtype=Float32, value=0.25)), AbstractScalar(Type: Bool, Value: true, Shape: NoShape), Prim: S_Prim_AllGather, Prim: S_Prim_AllReduce, )}, AbstractTuple{ element[0]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[1]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[2]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[3]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[4]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[5]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[6]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[7]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), sequence_nodes: {ValueNode (true, true, true, true, true, true, true, true), elements_use_flags: {ptr: 0x2b546690, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} , AbstractTuple{ element[0]: AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[1]: AbstractTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[2]: AbstractTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[3]: AbstractTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[4]: AbstractTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[5]: AbstractTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[6]: AbstractTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[7]: AbstractTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), sequence_nodes: {@map_245:grads{[0]: ValueNode MakeTuple, [1]: grads, [2]: grads, [3]: grads, [4]: grads, [5]: grads, [6]: grads, [7]: grads, [8]: grads}, elements_use_flags: {ptr: 0x2b525010, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: map_246 [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.656.435 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 75 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.657.580 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 76 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.658.481 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 77 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.659.559 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 78 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.660.624 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 79 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.661.726 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 80 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.662.354 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensors_allreduce_248] Added global python symbol: {F : } [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.662.818 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 81 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.663.453 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x252d6f20, value: Tensor(shape=[], dtype=Float32, value=0.25)), AbstractScalar(Type: Bool, Value: true, Shape: NoShape), Prim: S_Prim_AllGather, Prim: S_Prim_AllReduce, AbstractScalar(Type: Bool, Value: true, Shape: NoShape), AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny)}, g: _tensors_allreduce_248 [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.663.789 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 82 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.665.255 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 83 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.666.567 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_249:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_249:CNode_250{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.666.569 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 84 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.666.659 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_249:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_249:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.667.338 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 85 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.668.426 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 86 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.668.983 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_251:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_251:CNode_252{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.669.068 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_251:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_251:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.669.497 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 87 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.670.622 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 88 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.671.415 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_253:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_253:CNode_254{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.671.500 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_253:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_253:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.671.760 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 89 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.672.791 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 90 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.673.949 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_255:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_255:CNode_256{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.674.035 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_255:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_255:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.674.393 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 91 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.675.680 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 92 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.676.317 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_257:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_257:CNode_258{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.676.402 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_257:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_257:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.676.560 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 93 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.677.615 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 94 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.678.750 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 95 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.678.793 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_259:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_259:CNode_260{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.678.879 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_259:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_259:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.679.764 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 96 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.680.885 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 97 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.681.164 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_261:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_261:CNode_262{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.681.251 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_261:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_261:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.682.127 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 98 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.683.725 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 99 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.683.729 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_263:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_263:CNode_264{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.683.857 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_263:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_263:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.684.571 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: S-signature()}, AbstractTuple{ element[0]: AbstractType(Value: Float32), element[1]: AbstractType(Value: Float32), element[2]: AbstractType(Value: Float32), element[3]: AbstractType(Value: Float32), element[4]: AbstractType(Value: Float32), element[5]: AbstractType(Value: Float32), element[6]: AbstractType(Value: Float32), element[7]: AbstractType(Value: Float32), sequence_nodes: {@map_265:datatypes{[0]: ValueNode MakeTuple, [1]: datatypes, [2]: datatypes, [3]: datatypes, [4]: datatypes, [5]: datatypes, [6]: datatypes, [7]: datatypes, [8]: datatypes}, elements_use_flags: {ptr: 0x2b4e18e0, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} , AbstractTuple{ element[0]: AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[1]: AbstractTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[2]: AbstractTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[3]: AbstractTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[4]: AbstractTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[5]: AbstractTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[6]: AbstractTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[7]: AbstractTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), sequence_nodes: {@map_266:new_grad{[0]: ValueNode MakeTuple, [1]: new_grad, [2]: new_grad, [3]: new_grad, [4]: new_grad, [5]: new_grad, [6]: new_grad, [7]: new_grad, [8]: new_grad}, elements_use_flags: {ptr: 0x2ab48c10, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: map_267 [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:31.684.906 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 100 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.694.432 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: S-signature(Prim: S_Prim_ApplyMomentum, AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), )}, AbstractTuple{ element[0]: AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[1]: AbstractTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[2]: AbstractTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[3]: AbstractTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[4]: AbstractTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[5]: AbstractTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[6]: AbstractTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), element[7]: AbstractTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), sequence_nodes: {@map_270:new_grad{[0]: ValueNode MakeTuple, [1]: new_grad, [2]: new_grad, [3]: new_grad, [4]: new_grad, [5]: new_grad, [6]: new_grad, [7]: new_grad, [8]: new_grad}, elements_use_flags: {ptr: 0x2abcafe0, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} , AbstractTuple{ element[0]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[1]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[2]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[3]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[4]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[5]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[6]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[7]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), sequence_nodes: {@mindspore_nn_optim_momentum_Momentum_construct_271:CNode_272{[0]: ValueNode MakeTuple, [1]: param_conv1.weight, [2]: param_conv2.weight, [3]: param_fc1.weight, [4]: param_fc1.bias, [5]: param_fc2.weight, [6]: param_fc2.bias, [7]: param_fc3.weight, [8]: param_fc3.bias}, elements_use_flags: {ptr: 0x2abe0950, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} , AbstractTuple{ element[0]: AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[1]: AbstractRefTensor(key: moments.conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[2]: AbstractRefTensor(key: moments.fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[3]: AbstractRefTensor(key: moments.fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[4]: AbstractRefTensor(key: moments.fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[5]: AbstractRefTensor(key: moments.fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[6]: AbstractRefTensor(key: moments.fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), element[7]: AbstractRefTensor(key: moments.fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), sequence_nodes: {@mindspore_nn_optim_momentum_Momentum_construct_271:CNode_273{[0]: ValueNode MakeTuple, [1]: param_moments.conv1.weight, [2]: param_moments.conv2.weight, [3]: param_moments.fc1.weight, [4]: param_moments.fc1.bias, [5]: param_moments.fc2.weight, [6]: param_moments.fc2.bias, [7]: param_moments.fc3.weight, [8]: param_moments.fc3.bias}, elements_use_flags: {ptr: 0x2abe11f0, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: hyper_map_274 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.695.101 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: S-signature(Prim: S_Prim_ApplyMomentum, AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), )}, AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny)}, g: hyper_map_275 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.699.507 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensor_run_opt_ext_277] Added global python symbol: {F : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.699.760 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1261] ParseNameConstant] The NameConstant is bool:True [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.700.251 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{Prim: S_Prim_ApplyMomentum, AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny)}, g: _tensor_run_opt_ext_277 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.709.107 [mindspore/ccsrc/pipeline/jit/ps/action.cc:282] AbstractAnalyze] function call depth: 0, simulate call depth: 0 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.709.333 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:222] Run] Specialize set top func graph context: {FuncGraph: mindspore_train_dataset_helper__DataWrapper_construct_103 Args: [0]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [1]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [2]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [3]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [4]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [5]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [6]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [7]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [8]: AbstractRefTensor(key: global_step ref_value: AbstractRefTensor(shape: (1), element: AbstractScalar(Type: Int32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [9]: AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [10]: AbstractRefTensor(key: moments.conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [11]: AbstractRefTensor(key: moments.fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [12]: AbstractRefTensor(key: moments.fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [13]: AbstractRefTensor(key: moments.fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [14]: AbstractRefTensor(key: moments.fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [15]: AbstractRefTensor(key: moments.fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [16]: AbstractRefTensor(key: moments.fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [17]: AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [18]: AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), Parent: } [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.709.529 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @mindspore_train_dataset_helper__DataWrapper_construct_103:CNode_278{[0]: ValueNode MetaFuncGraph-unpack_call.279, [1]: ValueNode mindspore_nn_wrap_cell_wrapper_TrainOneStepCell_construct_280, [2]: outputs}, flag: 1 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.710.052 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @UnpackCall_281:CNode_278{[0]: param_282, [1]: CNode_278, [2]: CNode_278}, flag: 1 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.710.426 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @mindspore_nn_wrap_cell_wrapper_TrainOneStepCell_construct_283:CNode_284{[0]: ValueNode ✓mindspore_nn_wrap_cell_wrapper_TrainOneStepCell_construct_285}, flag: 1 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.710.659 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @✓mindspore_nn_wrap_cell_wrapper_TrainOneStepCell_construct_285:CNode_286{[0]: ValueNode MetaFuncGraph-unpack_call.287, [1]: ValueNode _no_sens_impl_288, [2]: CNode_289}, flag: 1 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.710.933 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @UnpackCall_290:CNode_286{[0]: param_291, [1]: CNode_286, [2]: CNode_286}, flag: 1 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.711.112 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @_no_sens_impl_216:CNode_292{[0]: ValueNode ✗_no_sens_impl_293}, flag: 1 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.711.228 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @✗_no_sens_impl_293:CNode_294{[0]: ValueNode ↓_no_sens_impl_295}, flag: 1 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.711.313 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @_no_sens_impl_216:loss{[0]: ValueNode S_Prim_Depend, [1]: loss, [2]: CNode_7}, flag: 1 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.711.370 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @_no_sens_impl_216:CNode_7{[0]: ValueNode mindspore_nn_optim_momentum_Momentum_construct_271, [1]: grads}, flag: 1 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.717.460 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @mindspore_nn_optim_momentum_Momentum_construct_271:CNode_17{[0]: ValueNode Depend, [1]: CNode_18, [2]: CNode_19}, flag: 1 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.746.520 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end abstract_specialize action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.746.594 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pack_expand action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.747.027 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pack_expand action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.747.071 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start auto_monad action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.750.450 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end auto_monad action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.750.501 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start inline action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.750.519 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end inline action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.750.538 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pre_auto_parallel action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.750.563 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pre_auto_parallel action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.750.582 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pipeline_split action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.750.596 [mindspore/ccsrc/pipeline/jit/ps/pipeline_split.cc:247] PipelineSplit] Only auto_parallel and semi_auto_parallel support pipeline split. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.750.607 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pipeline_split action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.750.624 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start optimize action. [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.761.976 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [fill_296] Added global python symbol: {cast_ : Prim[Cast]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.762.238 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] fill_296 update var `value` with node @fill_296:value{[0]: CNode_297, [1]: param_value, [2]: param_type} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.762.510 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [fill_296] Added global python symbol: {fillv2_ : Prim[FillV2]} [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.787.293 [mindspore/ccsrc/frontend/parallel/pynative_shard/pynative_shard.cc:334] PynativeShard] Only auto_parallel and semi_auto_parallel support pynative shard [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.787.366 [mindspore/ccsrc/frontend/parallel/step_parallel.cc:3009] StepParallel] Strategies would be ignored in data_parallel, shard() only valid in [semi_]auto_parallel. [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.804.678 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bprop_depend_298] Added global python symbol: {C : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:37:31.809.553 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bprop_load_299] Added global python symbol: {C : } [INFO] OPTIMIZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.818.481 [mindspore/ccsrc/frontend/optimizer/ad/bprop_utils.cc:70] GetBprop] Fail to find bprop function for UpdateState. fn: None [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.834.882 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractUMonad(ValueAny)}, g: _zeros_like_u_monad_302 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.849.331 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractUMonad(ValueAny), AbstractUMonad(ValueAny)}, g: hyper_map_303 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.851.957 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractUMonad(ValueAny), AbstractUMonad(ValueAny)}, g: _add_umonad_umonad_304 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.854.781 [mindspore/ccsrc/pipeline/jit/ps/action.cc:282] AbstractAnalyze] function call depth: 0, simulate call depth: 0 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.854.991 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:222] Run] Specialize set top func graph context: {FuncGraph: 1_mindspore_train_dataset_helper__DataWrapper_construct_300 Args: [0]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [1]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [2]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [3]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [4]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [5]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [6]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [7]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [8]: AbstractRefTensor(key: global_step ref_value: AbstractRefTensor(shape: (1), element: AbstractScalar(Type: Int32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [9]: AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [10]: AbstractRefTensor(key: moments.conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [11]: AbstractRefTensor(key: moments.fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [12]: AbstractRefTensor(key: moments.fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [13]: AbstractRefTensor(key: moments.fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [14]: AbstractRefTensor(key: moments.fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [15]: AbstractRefTensor(key: moments.fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [16]: AbstractRefTensor(key: moments.fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [17]: AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [18]: AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), Parent: } [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.900.232 [mindspore/ccsrc/frontend/parallel/pynative_shard/pynative_shard.cc:334] PynativeShard] Only auto_parallel and semi_auto_parallel support pynative shard [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.906.475 [mindspore/ccsrc/pipeline/jit/ps/action.cc:282] AbstractAnalyze] function call depth: 0, simulate call depth: 0 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.906.861 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:222] Run] Specialize set top func graph context: {FuncGraph: 251_1_mindspore_train_dataset_helper__DataWrapper_construct_315 Args: [0]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [1]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [2]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [3]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [4]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [5]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [6]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [7]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [8]: AbstractRefTensor(key: global_step ref_value: AbstractRefTensor(shape: (1), element: AbstractScalar(Type: Int32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [9]: AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [10]: AbstractRefTensor(key: moments.conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [11]: AbstractRefTensor(key: moments.fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [12]: AbstractRefTensor(key: moments.fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [13]: AbstractRefTensor(key: moments.fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [14]: AbstractRefTensor(key: moments.fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [15]: AbstractRefTensor(key: moments.fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [16]: AbstractRefTensor(key: moments.fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [17]: AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [18]: AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), Parent: } [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.913.875 [mindspore/ccsrc/frontend/parallel/pynative_shard/pynative_shard.cc:334] PynativeShard] Only auto_parallel and semi_auto_parallel support pynative shard [INFO] OPTIMIZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.919.405 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] OPTIMIZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.920.483 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] OPTIMIZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.921.324 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.921.481 [mindspore/ccsrc/frontend/parallel/cache_embedding/cache_embedding.cc:702] AddCacheEmbedding] Parameters are all not cache enable. [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.922.532 [mindspore/ccsrc/frontend/parallel/pass/assign_add_opt.cc:120] AssignAddOpt] parallel::g_device_manager is not initialized. [INFO] OPTIMIZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.922.596 [mindspore/ccsrc/frontend/optimizer/comm_op_reuse_tag.cc:59] AddCommOpReuseTag] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.922.617 [mindspore/ccsrc/frontend/parallel/pass/overlap_opt_shard_in_pipeline.cc:70] OverlapOptShardInPipeline] parallel::g_device_manager is not initialized. [INFO] OPTIMIZER(164043,ffffa1f54440,python):2024-01-10-11:37:31.922.634 [mindspore/ccsrc/frontend/optimizer/grouped_pairwise_exchange_alltoall.cc:673] SetGroupedPairwiseExchangeAllToAll] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.922.655 [mindspore/ccsrc/frontend/parallel/pass/overlap_gradmatmul_and_gradallreduce.cc:358] OverlapGradMatmulAndGradAllreduce] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.922.673 [mindspore/ccsrc/frontend/parallel/pass/split_matmul_comm_elementwise_fp.cc:184] SplitMatmulCommElementwiseFp] SplitMatmulCommElementwiseFp is only support under [semi_]auto_parallel, skip it. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.922.702 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end optimize action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.922.724 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start auto_monad_reorder action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.923.073 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end auto_monad_reorder action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.923.108 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start get_jit_bprop_graph action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.923.125 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end get_jit_bprop_graph action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.923.144 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start eliminate_special_op_node action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.924.007 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end eliminate_special_op_node action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.924.055 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start validate action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.924.334 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end validate action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.924.367 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start distribtued_split action. [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.924.400 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:372] GenerateStrategy] Current parallel mode is data_parallel [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.924.414 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:384] GenerateStrategy] Generated distributed strategy is 1 [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.924.806 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:1270] Run] All nodes are on this precoess so there's no need to build and split distributed graph. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.924.840 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end distribtued_split action. [INFO] PROFILER(164043,ffffa1f54440,python):2024-01-10-11:37:31.924.882 [mindspore/ccsrc/plugin/device/ascend/hal/profiler/parallel_strategy_profiling.cc:48] IsProfilingParallelStrategyEnabled] Profiling parallel strategy is disabled. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.924.902 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start task_emit action. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.925.304 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.925.330 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:31.925.343 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1252] SetRunMode] Run graph mode with kernel by kernel by configuration. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:37:31.925.413 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:546] CompileGraphs] Status record: start compile function graph: 381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.925.600 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unify_mindir_pm_0_erase_invalid_micro_depend in 1.94 us [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.926.740 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.926.775 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.141 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.163 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.189 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.204 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.222 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.234 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.250 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.262 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.278 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.300 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.317 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.329 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.343 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.355 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.369 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.381 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.396 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.407 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.424 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.437 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.452 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.463 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.490 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.503 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.520 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.532 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.548 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.559 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.575 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.586 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.601 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.613 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.636 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.648 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.664 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.676 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.692 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.703 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.718 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.729 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.744 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.755 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.770 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.780 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.795 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.807 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.822 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.833 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.848 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.858 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.873 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.884 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.899 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.910 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.926 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.943 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.962 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.974 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.927.990 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.002 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.017 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.029 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.045 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.058 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.074 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.086 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.101 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.114 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.130 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.142 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.157 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.169 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.185 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.197 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.212 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.224 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.239 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.251 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.266 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.293 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.308 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.321 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.337 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.349 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.364 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.376 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.391 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.402 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.417 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.429 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.443 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.454 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.469 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.481 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.497 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.508 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.524 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.535 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.550 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.562 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.577 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.588 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.609 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.621 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.636 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.648 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.664 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.676 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.694 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.705 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.720 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.731 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.747 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.758 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.773 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.784 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.799 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.811 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.825 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.837 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.851 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.863 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.878 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.889 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.904 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.919 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.935 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.947 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.962 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.973 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.928.990 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.001 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.016 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.027 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.043 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.055 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.070 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.081 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.096 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.107 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.122 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.134 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.149 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.161 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.177 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.189 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.203 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.214 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.230 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.246 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.262 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.275 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.290 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.301 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.318 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.330 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.344 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.355 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.371 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.382 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.397 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.408 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.424 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.434 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.449 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.460 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.474 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.486 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.501 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.513 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.528 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.539 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.559 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.571 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.586 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.597 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.612 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.624 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.639 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.651 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.666 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.677 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.700 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.714 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.730 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.741 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.756 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.768 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.782 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.794 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.807 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.819 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.833 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.844 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.919 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:727] CompileGraph] Compile graph: 381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316, Split segments size: 2 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:37:31.929.975 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:762] CompileGraphFromSegment] Compile normal segment, the first node: @381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316:equiv_CNode_317{[0]: ValueNode Load, [1]: param_fc3.bias, [2]: ValueNode U} [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:31.930.424 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:421] CompileGraph] Status record: start compile graph. [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.930.470 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:1940] ConstructKernelGraph] Create graph: 1 [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.931.201 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:872] CreateNewValueNode] Data sync for Tensor Tensor(shape=[1], dtype=Int32, value=[1]) [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.931.390 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:872] CreateNewValueNode] Data sync for Tensor Tensor(shape=[], dtype=Float32, value=0.25) [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.932.790 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:2697] ConstructOutput] Output:@381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316:CNode_278{[0]: ValueNode Depend, [1]: CNode_278, [2]: CNode_318} [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.934.633 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:193] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:1 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.935.093 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 88.05 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.935.687 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 545.38 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.936.589 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:154] CommonUnifyMindIR] start common unify mindir opt graph:1 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.936.919 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.937.340 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 416.45 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.937.374 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.937.415 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 37.65 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.937.434 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.937.451 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 15.05 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.937.465 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.937.996 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 516.77 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.938.957 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:79] BackendCommonOptimization] Status record: start common optimization. graph id: 1 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.939.293 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: add_dynamic_shape_attr [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.940.091 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_0_add_dynamic_shape_attr in 789.68 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.940.129 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.940.183 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_1_convert_dynamic_broadcast_to in 50.7 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.940.771 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_2_convert_const_input_to_attr in 556.55 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.941.280 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_3_custom_op_const_input_to_attr in 467.22 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.941.760 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 439.1 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.943.182 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 1374.66 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.943.241 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 16.97 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.943.624 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_7_flatten_concat_fission in 349.97 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.943.987 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 327.55 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.944.342 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_9_broadcast_to_fusion in 322.28 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.944.848 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 468.8 us [WARNING] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.945.798 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:92] BackendCommonOptimization] [PROF]backend_common_optimization costs 6841 usec. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.945.834 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:99] BackendCommonOptimization] Status record: end common optimization. graph id: 1 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.946.592 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_0_renorm_split in 355.38 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.946.627 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.947.568 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_1_reduce_axis_update in 929.07 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.947.600 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim HistogramFixedWidth not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.947.631 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_2_histogram_fixed_width_fusion in 29.69 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.947.649 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim ClipByNorm not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.947.665 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_3_clip_by_norm_fission in 15.73 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.947.680 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.947.694 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_4_tensor_arry_add_flow_cond1 in 13.14 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.947.707 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayGather not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.947.720 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_5_tensor_arry_add_flow_cond2 in 11.68 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.947.733 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.947.746 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_6_ge_tensor_arry_cast_index in 11.59 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.947.760 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArray not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.947.773 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_7_tensor_array_prepare in 11.93 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.947.786 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.947.824 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_8_space_to_batch_nd_attr_update in 34.7 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.947.842 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.947.871 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_9_batch_to_space_nd_attr_update in 26.76 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.947.886 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: avg_pool_grad_unify_mindir [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.947.922 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_10_avg_pool_grad_unify_mindir in 32.84 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.948.302 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_11_adam_weight_decay_unify_mindir in 353.47 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.948.665 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_12_cdist_fission in 328.59 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.949.021 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_13_cdist_grad_fission in 323.69 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.949.494 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_14_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 437 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.950.783 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_15_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 1229.29 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.951.226 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_16_sparse_softmax_cross_entropy_with_logits_unify_mindir in 391.12 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.951.627 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_17_dropout_unify_mindir1 in 358.75 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.951.657 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.036 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_18_dropoutgrad_unify_mindir in 368.72 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.067 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.086 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_19_neighbor_exchange_unify_mindir in 18.8 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.101 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.115 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_20_neighbor_exchange_v2_unify_mindir in 12.88 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.129 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.143 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_21_neighbor_exchange_v2_grad_unify_mindir in 11.96 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.156 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.169 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_22_all_to_all_unify_mindir in 11.85 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.184 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim QuantDTypeCast not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.198 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_23_quant_dtype_cst_adjust in 14.37 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.213 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim FSEDecode not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.227 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_24_fse_decode_adjust in 13.85 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.241 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.254 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_25_bn_split in 12.01 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.278 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.343 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_26_bn_grad_unify_mindir in 61.01 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.361 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.375 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_27_bn_grad_split in 12.92 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.388 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.402 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_28_batchnorm_to_bninfer in 11.58 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.415 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.428 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_29_batchnormgrad_to_bninfergrad in 11.61 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.440 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.952.454 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_30_batch_norm_grad_infer_fission in 11.41 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.953.185 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_31_ascend_mindir_op_adapter in 702.13 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.953.223 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_32_DropoutGenMask in 1.34 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.953.623 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_33_lamb_fission_ge in 372.76 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.954.564 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_34_print_insert_placeholder_for_tensor_name in 902.62 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.954.999 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_35_getnext_for_ge in 389.32 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.955.395 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_36_sync_bn_split in 353.35 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.955.779 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_37_sync_bn_grad_split in 347.22 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.956.164 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_38_adaptive_max_pool2d_ge_fusion in 347.97 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.956.548 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_39_avg_pool_grad_for_ge in 347.51 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.956.578 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:260] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.957.530 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_40_FlashAttentionFusionV1 in 943.79 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.957.564 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:374] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.958.561 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_41_FlashAttentionFusionV1 in 988.53 us [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.960.692 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:37:31.960.729 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:31.960.745 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:132] GetRunMode] RunMode::kKernelMode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.962.096 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 1298.77 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.964.000 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_0_seed_adapter in 1198.2 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.964.040 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: env_op_attr_update [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.964.756 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_1_env_op_attr_update in 699.1 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.965.175 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_2_process call inline in 372.08 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.965.414 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_3_expander_fallback in 191.43 us [WARNING] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.966.114 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:139] GEBackendOptimizeACL] [PROF]ascend_backend_optimize_acl costs 3330 usec. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.966.176 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] GetNext is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.966.623 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2D is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.966.926 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPool is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.967.069 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2D is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.967.229 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPool is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.967.398 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.967.778 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.968.009 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.968.232 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] SoftmaxCrossEntropyWithLogits is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.968.686 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.968.817 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.968.939 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.969.197 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.969.485 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.969.585 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.969.682 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.969.959 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.970.195 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.970.289 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.970.405 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.970.632 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.970.865 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPoolGrad is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.971.113 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AssignAdd is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.971.228 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2DBackpropInput is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.971.412 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2DBackpropFilter is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.971.588 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.971.773 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.971.953 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPoolGrad is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.972.136 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2DBackpropFilter is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.972.268 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.972.454 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.972.573 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.972.687 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.972.799 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.972.909 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.973.019 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.973.180 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.973.778 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 175.65 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.974.756 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 927.15 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.975.675 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 874.01 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.977.277 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 1551.77 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.978.836 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_4_insert_type_transform_op in 1502.94 us [WARNING] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.979.496 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:179] GEBackendOptimizeACLAfterKernelSelect] [PROF]ascend_backend_optimize_acl_after_kernel_select costs 5943 usec. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.979.608 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_0_DropoutGenMask in 0.77 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.979.967 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_1_cse in 314.61 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.981.184 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_redundant_op in 1172.16 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.981.400 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_3_eliminate_maketuple_getitem in 158.89 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.981.435 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_4_MergeTransData in 1.17 us [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.982.517 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive GetNext [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:31.982.708 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:467] ConvertAny] Value: ValueTuple [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.982.799 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive GetNext [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.982.827 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.982.941 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.982.967 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive OneHot [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.983.135 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive OneHot [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.983.163 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2D [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:31.983.231 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:31.983.271 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:31.983.300 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.983.386 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2D [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.983.409 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.983.483 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.983.505 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPool [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.983.647 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPool [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.983.673 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2D [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:31.983.729 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:31.983.762 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:31.983.800 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.983.877 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2D [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.983.901 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.983.978 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.984.001 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPool [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.984.091 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPool [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.984.114 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Flatten [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.984.199 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Flatten [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.984.223 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.984.345 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.984.371 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.984.520 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.984.548 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.984.618 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.984.642 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.984.737 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.984.762 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.984.863 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.984.888 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.984.955 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.984.987 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.985.075 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.985.099 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.985.196 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.985.220 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.985.302 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.985.325 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive SoftmaxCrossEntropyWithLogits [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.985.436 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive SoftmaxCrossEntropyWithLogits [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.985.464 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.985.547 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.985.570 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.985.652 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.985.675 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReduceMean [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.985.850 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReduceMean [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.985.876 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.985.982 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.986.006 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.986.101 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.986.126 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.986.216 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.986.325 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.986.350 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.986.411 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.986.495 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.986.518 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReluGrad [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.986.596 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReluGrad [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.986.618 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.986.698 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.986.720 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.986.799 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.986.822 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.986.881 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.986.960 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.986.982 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.987.066 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.987.092 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.987.149 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.987.225 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.987.246 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReluGrad [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.987.333 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReluGrad [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.987.357 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.987.437 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.987.459 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.987.551 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.987.575 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.987.632 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.987.706 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.987.728 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.987.810 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.987.834 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.987.893 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.987.969 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.987.991 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.988.086 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.988.110 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPoolGrad [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.988.239 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPoolGrad [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.988.264 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReluGrad [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.988.344 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReluGrad [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.988.368 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive AssignAdd [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.988.461 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive AssignAdd [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.988.485 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2DBackpropInput [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:31.988.563 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:31.988.594 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.988.678 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2DBackpropInput [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.988.701 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2DBackpropFilter [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:31.988.763 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:31.988.799 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:31.988.828 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.988.910 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2DBackpropFilter [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.988.936 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.988.998 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.989.078 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.989.101 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.989.237 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.989.262 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPoolGrad [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.989.370 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPoolGrad [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.989.398 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReluGrad [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.989.477 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReluGrad [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.989.500 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2DBackpropFilter [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:31.989.575 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:31.989.614 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:37:31.989.641 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.989.742 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2DBackpropFilter [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.989.768 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.989.831 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.989.908 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.989.929 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.990.034 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.990.060 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.990.159 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.990.182 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.990.278 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.990.301 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.990.393 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.990.416 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.990.517 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.990.543 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.990.638 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.990.664 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.990.749 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.990.774 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.990.872 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [WARNING] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:31.990.893 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:262] CreateKernel] [PROF]create_kernel costs 8417 usec. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:31.991.154 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/Reshape-op0, index: 0 to input Default/GetNext-op1, index: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:31.991.194 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1, index: 0 to input Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2, index: 0 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:31.991.223 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/Reshape-op1, index: 0 to input Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1, index: 0 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:31.991.249 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/Reshape-op2, index: 0 to input Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/Mul-op0, index: 0 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:31.991.272 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/gradFlatten-expand/Reshape-op1, index: 0 to input Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op6, index: 0 [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.991.290 [mindspore/ccsrc/backend/common/session/session_basic.cc:1140] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 1 start [INFO] DEBUG(164043,ffffa1f54440,python):2024-01-10-11:37:31.991.303 [mindspore/ccsrc/debug/summary/summary.cc:33] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 1 start [INFO] DEBUG(164043,ffffa1f54440,python):2024-01-10-11:37:31.991.321 [mindspore/ccsrc/debug/summary/summary.cc:54] RecurseSetSummaryNodesForAllGraphs] The total summary nodes is: 0 for graph: 1 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:31.991.947 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8's input.Send node Default/StreamSend-op0 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op7, recv node Default/StreamRecv-op0 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:31.992.019 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8's output.Send node Default/StreamSend-op1 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8, recv node Default/StreamRecv-op1 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op8 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:31.992.090 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9's input.Send node Default/StreamSend-op2 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op3, recv node Default/StreamRecv-op2 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:31.992.139 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9's output.Send node Default/StreamSend-op3 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9, recv node Default/StreamRecv-op3 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op9 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:31.992.196 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10's input.Send node Default/StreamSend-op4 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op8, recv node Default/StreamRecv-op4 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:31.992.242 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10's output.Send node Default/StreamSend-op5 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10, recv node Default/StreamRecv-op5 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op10 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:31.992.299 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11's input.Send node Default/StreamSend-op6 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op4, recv node Default/StreamRecv-op6 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:31.992.346 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11's output.Send node Default/StreamSend-op7 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11, recv node Default/StreamRecv-op7 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op11 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:31.992.403 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12's input.Send node Default/StreamSend-op8 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op9, recv node Default/StreamRecv-op8 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:31.992.461 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12's output.Send node Default/StreamSend-op9 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12, recv node Default/StreamRecv-op9 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op12 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:31.992.513 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13's input.Send node Default/StreamSend-op10 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op5, recv node Default/StreamRecv-op10 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:31.992.560 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13's output.Send node Default/StreamSend-op11 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13, recv node Default/StreamRecv-op11 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op13 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:31.992.625 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14's input.Send node Default/StreamSend-op12 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/gradConv2D-expand/Conv2DBackpropFilter-op1, recv node Default/StreamRecv-op12 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:31.992.671 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14's output.Send node Default/StreamSend-op13 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14, recv node Default/StreamRecv-op13 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op14 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:31.992.728 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15's input.Send node Default/StreamSend-op14 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv1-Conv2d/gradConv2D-expand/Conv2DBackpropFilter-op1, recv node Default/StreamRecv-op14 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:31.992.775 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15's output.Send node Default/StreamSend-op15 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15, recv node Default/StreamRecv-op15 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op15 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.992.907 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.992.939 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:0 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.992.963 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.992.981 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:0 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.002 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.020 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:1 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.042 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.060 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:2 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.081 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.099 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:2 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.120 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.137 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:3 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.155 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.172 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:1 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.195 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.211 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:4 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.230 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.246 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:4 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.265 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.282 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:5 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.299 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.316 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:3 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.335 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.351 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:6 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.378 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.395 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:6 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.413 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.428 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:7 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.448 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.464 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:5 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.485 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.502 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:8 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.519 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.535 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:8 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.554 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.569 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:9 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.587 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.601 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:7 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.619 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.634 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:10 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.651 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.666 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:10 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.684 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.728 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:11 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.746 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.762 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:9 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.784 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.800 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:12 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.826 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.842 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:12 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.860 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.874 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:13 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.892 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.908 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:11 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.930 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.946 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:14 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.964 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.993.981 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:14 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.000 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.016 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:15 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.034 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.050 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:13 [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.072 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.088 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:15 [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.107 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:891] PrintGraphExecuteOrder] Graph 1 execution order: [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.151 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[0], node name[Default/GetNext-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:outputs{[0]: ValueNode GetNext}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.210 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[1], node name[Default/Reshape-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_Reshape, [1]: CNode_278, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[32])}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.277 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[2], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/OneHot-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_OneHot, [1]: 319, [2]: ValueNode Tensor(shape=[], dtype=Int64, value=10), [3]: ValueNode Tensor(shape=[], dtype=Float32, value=1), [4]: ValueNode Tensor(shape=[], dtype=Float32, value=0), [5]: ValueNode -1}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.342 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[3], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv1-Conv2d/Conv2D-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode Conv2D, [1]: CNode_278, [2]: equiv_CNode_320}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.390 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[4], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op4], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_ReLU, [1]: equiv_CNode_278}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.440 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[5], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op3], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode MaxPool, [1]: equiv_CNode_278}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.494 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[6], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/Conv2D-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode Conv2D, [1]: equiv_CNode_278, [2]: equiv_CNode_321}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.540 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[7], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op5], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_ReLU, [1]: equiv_CNode_278}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.586 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[8], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode MaxPool, [1]: equiv_CNode_278}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.640 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[9], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_Flatten, [1]: equiv_CNode_278}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.702 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[10], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/MatMul-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode MatMul, [1]: equiv_CNode_278, [2]: equiv_CNode_322}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.758 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[11], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/BiasAdd-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_BiasAdd, [1]: equiv_CNode_278, [2]: equiv_CNode_323, [3]: ValueNode 0}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.804 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[12], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op6], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_ReLU, [1]: equiv_CNode_278}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.865 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[13], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/MatMul-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode MatMul, [1]: equiv_CNode_278, [2]: equiv_CNode_324}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.921 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[14], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/BiasAdd-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_BiasAdd, [1]: equiv_CNode_278, [2]: equiv_CNode_325, [3]: ValueNode 0}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.994.966 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[15], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op7], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_ReLU, [1]: equiv_CNode_278}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.017 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[16], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/MatMul-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode MatMul, [1]: equiv_CNode_278, [2]: equiv_CNode_326}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.069 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[17], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_BiasAdd, [1]: equiv_CNode_278, [2]: equiv_CNode_317, [3]: ValueNode 0}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.119 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[18], node name[Default/Reshape-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_Reshape, [1]: equiv_CNode_278, [2]: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10])}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.158 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[19], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/SoftmaxCrossEntropyWithLogits-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode SoftmaxCrossEntropyWithLogits, [1]: 319, [2]: 319}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.197 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[20], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/Mul-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_Mul, [1]: 319, [2]: ValueNode Tensor(shape=[32, 1], dtype=Float32, value=[...])}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.238 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[21], node name[Default/Reshape-op2], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_Reshape, [1]: 319, [2]: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10])}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.289 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[22], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/ReduceMean-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_ReduceMean, [1]: 319, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), [3]: ValueNode false}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.331 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[23], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op10], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 319, [2]: equiv_CNode_326}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.373 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[24], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op7], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 319, [2]: equiv_CNode_278}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.401 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[25], node name[Default/StreamSend-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_328{[0]: ValueNode StreamSend}], event id[0] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.425 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[26], node name[Default/StreamRecv-op0], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_329{[0]: ValueNode StreamRecv}], event id[0] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.468 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[27], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 327}], group[hccl_world_group] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.499 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[28], node name[Default/StreamSend-op1], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_330{[0]: ValueNode StreamSend}], event id[1] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.536 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[29], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op3], logic id[4294967295], stream id[0], node info[@kernel_graph_1:331{[0]: ValueNode PrimFunc_BiasAddGrad, [1]: 319, [2]: ValueNode 0}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.564 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[30], node name[Default/StreamSend-op2], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_332{[0]: ValueNode StreamSend}], event id[2] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.590 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[31], node name[Default/StreamRecv-op2], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_333{[0]: ValueNode StreamRecv}], event id[2] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.637 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[32], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 331}], group[hccl_world_group] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.666 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[33], node name[Default/StreamSend-op3], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_334{[0]: ValueNode StreamSend}], event id[3] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.691 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[34], node name[Default/StreamRecv-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_335{[0]: ValueNode StreamRecv}], event id[1] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.747 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[35], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op8], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.793 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[36], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/gradReLU-expand/ReluGrad-op4], logic id[4294967295], stream id[0], node info[@kernel_graph_1:336{[0]: ValueNode PrimFunc_ReluGrad, [1]: 327, [2]: equiv_CNode_278}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.831 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[37], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op11], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 336, [2]: equiv_CNode_324}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.872 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[38], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op8], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 336, [2]: equiv_CNode_278}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.898 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[39], node name[Default/StreamSend-op4], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_337{[0]: ValueNode StreamSend}], event id[4] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.923 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[40], node name[Default/StreamRecv-op4], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_338{[0]: ValueNode StreamRecv}], event id[4] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.963 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[41], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 327}], group[hccl_world_group] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.995.992 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[42], node name[Default/StreamSend-op5], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_339{[0]: ValueNode StreamSend}], event id[5] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.032 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[43], node name[Default/StreamRecv-op3], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_340{[0]: ValueNode StreamRecv}], event id[3] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.087 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[44], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op9], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.121 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[45], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op4], logic id[4294967295], stream id[0], node info[@kernel_graph_1:331{[0]: ValueNode PrimFunc_BiasAddGrad, [1]: 336, [2]: ValueNode 0}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.150 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[46], node name[Default/StreamSend-op6], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_341{[0]: ValueNode StreamSend}], event id[6] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.174 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[47], node name[Default/StreamRecv-op6], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_342{[0]: ValueNode StreamRecv}], event id[6] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.213 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[48], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 331}], group[hccl_world_group] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.239 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[49], node name[Default/StreamSend-op7], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_343{[0]: ValueNode StreamSend}], event id[7] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.264 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[50], node name[Default/StreamRecv-op5], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_344{[0]: ValueNode StreamRecv}], event id[5] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.315 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[51], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op10], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.361 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[52], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/gradReLU-expand/ReluGrad-op5], logic id[4294967295], stream id[0], node info[@kernel_graph_1:336{[0]: ValueNode PrimFunc_ReluGrad, [1]: 327, [2]: equiv_CNode_278}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.397 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[53], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op6], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 336, [2]: equiv_CNode_322}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.456 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[54], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op9], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 336, [2]: equiv_CNode_278}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.481 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[55], node name[Default/StreamSend-op8], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_345{[0]: ValueNode StreamSend}], event id[8] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.506 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[56], node name[Default/StreamRecv-op8], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_346{[0]: ValueNode StreamRecv}], event id[8] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.546 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[57], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 327}], group[hccl_world_group] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.572 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[58], node name[Default/StreamSend-op9], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_347{[0]: ValueNode StreamSend}], event id[9] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.598 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[59], node name[Default/StreamRecv-op7], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_348{[0]: ValueNode StreamRecv}], event id[7] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.650 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[60], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op11], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.684 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[61], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op5], logic id[4294967295], stream id[0], node info[@kernel_graph_1:331{[0]: ValueNode PrimFunc_BiasAddGrad, [1]: 336, [2]: ValueNode 0}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.710 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[62], node name[Default/StreamSend-op10], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_349{[0]: ValueNode StreamSend}], event id[10] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.738 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[63], node name[Default/StreamRecv-op10], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_350{[0]: ValueNode StreamRecv}], event id[10] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.787 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[64], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 331}], group[hccl_world_group] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.814 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[65], node name[Default/StreamSend-op11], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_351{[0]: ValueNode StreamSend}], event id[11] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.840 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[66], node name[Default/StreamRecv-op9], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_352{[0]: ValueNode StreamRecv}], event id[9] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.894 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[67], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op12], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.933 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[68], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/gradFlatten-expand/Reshape-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:353{[0]: ValueNode PrimFunc_Reshape, [1]: 327, [2]: ValueNode (32, 16, 5, 5)}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.996.995 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[69], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/gradMaxPool-expand/MaxPoolGrad-op2], logic id[4294967295], stream id[0], node info[@kernel_graph_1:354{[0]: ValueNode MaxPoolGrad, [1]: equiv_CNode_278, [2]: equiv_CNode_278, [3]: 353}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.038 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[70], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/gradReLU-expand/ReluGrad-op6], logic id[4294967295], stream id[0], node info[@kernel_graph_1:336{[0]: ValueNode PrimFunc_ReluGrad, [1]: 354, [2]: equiv_CNode_278}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.090 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[71], node name[Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AssignAdd, [1]: param_global_step, [2]: ValueNode Tensor(shape=[1], dtype=Int32, value=[1]), [3]: CNode_355}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.133 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[72], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/gradConv2D-expand/Conv2DBackpropInput-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:356{[0]: ValueNode Conv2DBackpropInput, [1]: 336, [2]: equiv_CNode_321, [3]: ValueNode (32, 6, 14, 14)}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.184 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[73], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/gradConv2D-expand/Conv2DBackpropFilter-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:356{[0]: ValueNode Conv2DBackpropFilter, [1]: 336, [2]: equiv_CNode_278, [3]: ValueNode (16, 6, 5, 5)}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.220 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[74], node name[Default/StreamSend-op12], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_357{[0]: ValueNode StreamSend}], event id[12] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.246 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[75], node name[Default/StreamRecv-op12], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_358{[0]: ValueNode StreamRecv}], event id[12] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.286 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[76], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 356}], group[hccl_world_group] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.311 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[77], node name[Default/StreamSend-op13], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_359{[0]: ValueNode StreamSend}], event id[13] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.336 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[78], node name[Default/StreamRecv-op11], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_360{[0]: ValueNode StreamRecv}], event id[11] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.389 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[79], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op13], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.464 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[80], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc3.bias, [2]: param_moments.fc3.bias, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_361}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.522 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[81], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/gradMaxPool-expand/MaxPoolGrad-op3], logic id[4294967295], stream id[0], node info[@kernel_graph_1:354{[0]: ValueNode MaxPoolGrad, [1]: equiv_CNode_278, [2]: equiv_CNode_278, [3]: 356}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.567 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[82], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/gradReLU-expand/ReluGrad-op7], logic id[4294967295], stream id[0], node info[@kernel_graph_1:336{[0]: ValueNode PrimFunc_ReluGrad, [1]: 354, [2]: equiv_CNode_278}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.610 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[83], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv1-Conv2d/gradConv2D-expand/Conv2DBackpropFilter-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:362{[0]: ValueNode Conv2DBackpropFilter, [1]: 336, [2]: CNode_278, [3]: ValueNode (6, 1, 5, 5)}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.647 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[84], node name[Default/StreamSend-op14], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_363{[0]: ValueNode StreamSend}], event id[14] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.676 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[85], node name[Default/StreamRecv-op14], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_364{[0]: ValueNode StreamRecv}], event id[14] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.737 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[86], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 362}], group[hccl_world_group] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.766 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[87], node name[Default/StreamSend-op15], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_365{[0]: ValueNode StreamSend}], event id[15] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.791 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[88], node name[Default/StreamRecv-op13], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_366{[0]: ValueNode StreamRecv}], event id[13] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.851 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[89], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op14], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.997.927 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[90], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc3.weight, [2]: param_moments.fc3.weight, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_367}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.998.001 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[91], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc2.bias, [2]: param_moments.fc2.bias, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_368}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.998.072 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[92], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc2.weight, [2]: param_moments.fc2.weight, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_369}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.998.144 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[93], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc1.bias, [2]: param_moments.fc1.bias, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_370}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.998.226 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[94], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc1.weight, [2]: param_moments.fc1.weight, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_371}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.998.301 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[95], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_conv2.weight, [2]: param_moments.conv2.weight, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_372}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.998.327 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[96], node name[Default/StreamRecv-op15], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_373{[0]: ValueNode StreamRecv}], event id[15] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.998.383 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[97], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op15], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:31.998.454 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[98], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_conv1.weight, [2]: param_moments.conv1.weight, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_374}] [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.998.961 [mindspore/ccsrc/backend/common/somas/somas.cc:143] Assign] Start Somas Assign for graph 1 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.999.400 [mindspore/ccsrc/backend/common/somas/somas.cc:151] Assign] Somas Configure success, configuration info: Device Name: Ascend Run by execution order: 1 Enable debug log: 0 Debug log path: . [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:31.999.422 [mindspore/ccsrc/backend/common/somas/somas.cc:154] Assign] Start Initialize SOMAS Model [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.000.787 [mindspore/ccsrc/backend/common/somas/somas.cc:1065] GraphOutputProcess] Set 1 graph output tensors' aligned size to 0. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.001.461 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 1 output 2 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.001.498 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 9 output 10 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.001.524 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 18 output 19 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.001.541 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 22 output 23 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.001.587 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 40 output 47 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.001.635 [mindspore/ccsrc/backend/common/somas/somas.cc:1135] RefNodeProcess] Special Tensor total size: RefNode: input 107008 output 351828 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.001.709 [mindspore/ccsrc/backend/common/somas/somas.cc:555] InitSomasModel] Created 2 streams (0 groups), 99 nodes, 69 tensors, 5 union tensors lists, and 0 contiguous tensors lists [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.002.352 [mindspore/ccsrc/backend/common/somas/somas.cc:157] Assign] End Initialize SOMAS Model [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.002.370 [mindspore/ccsrc/backend/common/somas/somas.cc:176] Assign] Start Computing Conflict Matrix [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.002.383 [mindspore/ccsrc/backend/common/somas/somas.cc:1286] ComputeBasicMatrix] Start Conflict Computing (Bitset Model) [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.002.405 [mindspore/ccsrc/backend/common/somas/somas.cc:1291] ComputeBasicMatrix] Start Bitset [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.002.445 [mindspore/ccsrc/backend/common/somas/somas.cc:1299] ComputeBasicMatrix] Start Path Computing [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.002.470 [mindspore/ccsrc/backend/common/somas/somas.cc:1307] ComputeBasicMatrix] End Path Computing [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.002.481 [mindspore/ccsrc/backend/common/somas/somas.cc:1309] ComputeBasicMatrix] Start Tensor Relation Computing [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.002.554 [mindspore/ccsrc/backend/common/somas/somas.cc:1462] ComputeMultiTensorConflicts] Start Computing Conflicts Pairs, tensors list size is 60 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.002.629 [mindspore/ccsrc/backend/common/somas/somas.cc:1469] ComputeMultiTensorConflicts] End Computing Conflicts Pairs (time taken 0ms) [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.002.642 [mindspore/ccsrc/backend/common/somas/somas.cc:1367] ComputeBasicMatrix] End Basic Conflict Computing (Bitset Model)(time taken 0ms) [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.002.674 [mindspore/ccsrc/backend/common/somas/somas.cc:178] Assign] End Computing Conflict Matrix [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.002.686 [mindspore/ccsrc/backend/common/somas/somas.cc:1533] Solve] Somas Assign start... [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.002.718 [mindspore/ccsrc/backend/common/somas/somas.cc:1555] Solve] Start Solving [INFO] PRE_ACT(164043,fffea37fe0f0,python):2024-01-10-11:37:32.002.986 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 55 tensors [INFO] PRE_ACT(164043,fffec0ff90f0,python):2024-01-10-11:37:32.003.014 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 55 tensors [INFO] PRE_ACT(164043,fffea2ffd0f0,python):2024-01-10-11:37:32.003.022 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 55 tensors [INFO] PRE_ACT(164043,fffea3fff0f0,python):2024-01-10-11:37:32.003.030 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 55 tensors [INFO] PRE_ACT(164043,fffea37fe0f0,python):2024-01-10-11:37:32.003.164 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 2/4 2196992 Bytes (0.00204611 GB) Shared Objects size(>), index(<) smallest [INFO] PRE_ACT(164043,fffec0ff90f0,python):2024-01-10-11:37:32.003.211 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 1/4 2196992 Bytes (0.00204611 GB) Shared Objects size(>), index(<) bestfit [INFO] PRE_ACT(164043,fffea2ffd0f0,python):2024-01-10-11:37:32.003.217 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 4/4 2196992 Bytes (0.00204611 GB) Single Object size(>), index(<) smallest [INFO] PRE_ACT(164043,fffea3fff0f0,python):2024-01-10-11:37:32.003.231 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 3/4 2196992 Bytes (0.00204611 GB) Single Object size(>), index(<) bestfit [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.285 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:176] Solving] SOMAS SOLVER RESUME: [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.301 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:177] Solving] Best Solution:[1/4] [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.317 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:178] Solving] Best result:2196992 Bytes 0.00204611 GB (0.00204611 GB + 0 GB from lifelong tensors) [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.329 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:181] Solving] Best timing:0 ms [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.340 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:182] Solving] Best algorithm: Shared Objects [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.351 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:183] Solving] Best sorting strategy: size(>), index(<) [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.362 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:184] Solving] Best offset strategy: bestfit [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.372 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:185] Solving] Time elapsed: 0 ms [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.384 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:186] Solving] Spread:0 %% [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.449 [mindspore/ccsrc/backend/common/somas/somas.cc:1564] Solve] End Solving [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.524 [mindspore/ccsrc/backend/common/somas/somas.cc:2096] GenGraphStatisticInfo] Lower Bound: 2186752 (0.00203657 GB), Upper Bound: 4660224 (0.00434017 GB) [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.537 [mindspore/ccsrc/backend/common/somas/somas.cc:2099] GenGraphStatisticInfo] Total Dynamic Size (Upper Bound): 4660224 Theoretical Optimal Size (Lower Bound): 2186752 Total Workspace Size: 0 Total Communication Input Tensor Size: 248832 Total Communication Output Tensor Size: 0 Total LifeLong All Tensor Size: 0 Total LifeLong Start Tensor Size: 0 Total LifeLong End Tensor Size: 512 Reused Size(Allocate Size): 0 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.550 [mindspore/ccsrc/backend/common/somas/somas.cc:1583] Solve] Somas Assign end. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.691 [mindspore/ccsrc/backend/common/somas/somas.cc:380] UpdateSomasResultToGraph] Merged Block size: 12 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.706 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 0, offset: 1205248, size: 602624 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.718 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 1, offset: 602624, size: 602624 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.729 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 2, offset: 0, size: 602624 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.739 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 3, offset: 1807872, size: 192512 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.750 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 4, offset: 2000384, size: 131584 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.760 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 5, offset: 2131968, size: 40448 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.785 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 6, offset: 2182144, size: 9728 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.797 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 7, offset: 2172416, size: 9728 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.807 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 8, offset: 2191872, size: 3584 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.818 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 9, offset: 2196480, size: 512 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.830 [mindspore/ccsrc/backend/common/somas/somas.cc:189] Assign] Somas Allocate end. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.003.842 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:310] DoSomas] Somas allocate success for graph 1 somas size: 2196992 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.004.045 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:801] CreateDeviceAddress] Status record: start create device address. graph id: 1 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.004.757 [mindspore/ccsrc/runtime/device/device_address_utils.cc:454] CreateValueNodeDeviceAddress] No device address for value node:Default/data-17, debug name:ValueNode U [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.326 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1, index is 0; cur kernel is Default/Reshape-op1, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.379 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1, index is 0; cur kernel is Default/Reshape-op1, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.407 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/Mul-op0, index is 0; cur kernel is Default/Reshape-op2, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.434 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/Mul-op0, index is 0; cur kernel is Default/Reshape-op2, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.456 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc3.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.476 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc3.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.492 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is conv2.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.520 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is conv2.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.536 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is conv1.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.552 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is conv1.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.566 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc2.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.582 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc2.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.597 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc2.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.614 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc2.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.629 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc1.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.645 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc1.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.660 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc1.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.676 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc1.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.691 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc3.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.709 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc3.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.741 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/GetNext-op1, index is 1; cur kernel is Default/Reshape-op0, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.773 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/GetNext-op1, index is 1; cur kernel is Default/Reshape-op0, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.793 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is global_step, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.809 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is global_step, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.825 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2, index is 0; cur kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.852 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2, index is 0; cur kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.871 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op6, index is 0; cur kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/gradFlatten-expand/Reshape-op1, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.894 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op6, index is 0; cur kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/gradFlatten-expand/Reshape-op1, index is 0 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.007.913 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:808] CreateDeviceAddress] Status record: end create device address. graph id: 1 [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:32.008.109 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1129] CacheGraphOutputToFrontNodeWithIndex] Get graph backend output nodes. [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:32.008.148 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1137] CacheGraphOutputToFrontNodeWithIndex] Get graph front output nodes. [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:37:32.008.176 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1155] CacheGraphOutputToFrontNodeWithIndex] Backend output: Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/ReduceMean-op0 with index: 0 map to front node: Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits-op0 with index: 0 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.008.295 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:507] CompileGraph] Status record: end compile graph. graph id: 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:37:32.008.757 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:805] CompileGraphFromSegment] Compile cut segment, the cut node: @381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316:CNode_375{[0]: ValueNode Return, [1]: CNode_278} [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.008.943 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:521] Transform] Graph(kernel_graph_1) transforms actor begin, strategy:pipeline_with_execution_order [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.008.982 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2619] PersistDeviceTensorForValueNode] The device address is not exist: ValueNode_376(U) [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.009.198 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1101] BuildDataSourceActor] Create queue data source actor: kernel_graph_1_DeviceDSActor_1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.010.178 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1232] BuildLoopCountActor] Create loop count actor: kernel_graph_1_LoopCountActor [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.010.214 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1265] BuildOutputActor] Create output actor: kernel_graph_1_OutputActor [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.010.234 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1285] BuildDataPrepareActor] Create data prepare actor: kernel_graph_1_DataPrepareActor [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.010.369 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:866] CacheGraphOutputToActor] Cache graph 1 output node:Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/ReduceMean-op0 debug string:@kernel_graph_1:319{[0]: ValueNode PrimFunc_ReduceMean, [1]: 319, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), [3]: ValueNode false} with index:0 to actor:Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/ReduceMean-op0, from front node:Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits-op0 debug string:@381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316:equiv_CNode_278{[0]: ValueNode SparseSoftmaxCrossEntropyWithLogits, [1]: equiv_CNode_278, [2]: CNode_278} with index:0 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.010.391 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:809] AddSomasInfoForGraphOutput] The graph 1 output node:Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/ReduceMean-op0 with index: 0 somas enable or not: 1, somas offset: 2195456, aligned size: 512 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.010.415 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1518] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_1 start. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.010.560 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1520] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_1 end. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.011.617 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1 add input node monad device tensor store:global_step [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.011.659 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1 add input node monad device tensor store:Default/network-TrainOneStepCell/optimizer-Momentum/data-0 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.011.819 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.011.849 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.011.871 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 add input node monad device tensor store:fc3.bias [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.011.884 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 add input node monad device tensor store:moments.fc3.bias [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.011.897 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.011.908 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.088 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.115 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/MatMul-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.138 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op10, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.168 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 add input node monad device tensor store:fc3.weight [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.182 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 add input node monad device tensor store:moments.fc3.weight [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.194 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.205 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.259 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.284 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/BiasAdd-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.303 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 add input node monad device tensor store:fc2.bias [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.317 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 add input node monad device tensor store:moments.fc2.bias [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.329 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.340 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.392 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.425 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/MatMul-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.449 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op11, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.467 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 add input node monad device tensor store:fc2.weight [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.480 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 add input node monad device tensor store:moments.fc2.weight [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.491 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.502 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.552 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.576 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/BiasAdd-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.595 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 add input node monad device tensor store:fc1.bias [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.609 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 add input node monad device tensor store:moments.fc1.bias [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.621 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.639 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.692 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.717 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/MatMul-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.740 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op6, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.759 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 add input node monad device tensor store:fc1.weight [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.771 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 add input node monad device tensor store:moments.fc1.weight [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.782 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.793 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.842 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.867 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/Conv2D-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.890 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/gradConv2D-expand/Conv2DBackpropInput-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.916 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 add input node monad device tensor store:conv2.weight [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.928 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 add input node monad device tensor store:moments.conv2.weight [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.939 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.012.950 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.013.022 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.013.047 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv1-Conv2d/Conv2D-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.013.067 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 add input node monad device tensor store:conv1.weight [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.013.081 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 add input node monad device tensor store:moments.conv1.weight [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.013.092 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.013.103 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.013.117 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.013.148 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.013.173 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.013.194 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.013.214 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.013.235 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.013.255 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.013.275 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.014.130 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 1_actor_set_kernel_graph_1_memory_actor_insert in 22.27 us [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.014.170 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 2_actor_set_kernel_graph_1_invalid_data_arrow_elimination in 6.78 us [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.014.419 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 3_actor_set_kernel_graph_1_multi_actor_fusion in 224.51 us [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.014.465 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 4_actor_set_kernel_graph_1_batch_data_arrow_fusion in 21.4 us [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:37:32.014.487 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:554] Transform] Graph(kernel_graph_1) transforms actor end. [WARNING] VM(164043,ffffa1f54440,python):2024-01-10-11:37:32.015.968 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:612] CompileGraphs] [PROF]compile_func_graph costs 90523 usec. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:37:32.016.014 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:615] CompileGraphs] Status record: end compile function graph: 381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316, produce actor: kernel_graph_1 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:32.016.039 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end task_emit action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:32.016.073 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:268] SetLoopCount] Change vm_loop_flag to 0, set loop_size to 468 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:32.016.093 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start execute action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:32.016.111 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end execute action. TotalTime = 0.776818, [19] [parse]: 0.0113525 [symbol_resolve]: 0.251701, [1] [Cycle 1]: 0.25105, [1] [resolve]: 0.250952 [graph_reusing]: 7.144e-05 [meta_unpack_prepare]: 0.00118865 [pre_cconv]: 3.343e-05 [abstract_specialize]: 0.242838 [pack_expand]: 0.00046389 [auto_monad]: 0.00341409 [inline]: 3.31e-05 [pre_auto_parallel]: 3.765e-05 [pipeline_split]: 3.623e-05 [optimize]: 0.17209, [35] [py_interpret_to_execute]: 0.00112067 [rewriter_before_opt_a]: 0.00604817 [opt_a]: 0.157709, [3] [Cycle 1]: 0.125451, [30] [expand_dump_flag]: 6.807e-05 [switch_simplify]: 0.00106243 [a_1]: 0.0179413 [recompute_prepare]: 0.00015579 [updatestate_depend_eliminate]: 0.00080995 [updatestate_assign_eliminate]: 0.00018879 [updatestate_loads_eliminate]: 0.00047774 [parameter_eliminate]: 1.852e-05 [a_2]: 0.00298291 [accelerated_algorithm]: 9.516e-05 [pynative_shard]: 4.831e-05 [auto_parallel]: 5.07e-06 [parallel]: 4.312e-05 [merge_comm]: 7.998e-05 [allreduce_fusion]: 4.575e-05 [virtual_dataset]: 8.642e-05 [get_grad_eliminate_]: 7.026e-05 [virtual_output]: 6.78e-05 [merge_forward]: 0.00011993 [cell_reuse_recompute_pass]: 4.70005e-07 [cell_reuse_handle_not_recompute_node_pass]: 0.00020856 [meta_fg_expand]: 0.0333595 [after_resolve]: 0.00043522 [a_after_grad]: 0.0006406 [renormalize]: 0.0555984 [real_op_eliminate]: 0.00067447 [auto_monad_grad]: 0.00047822 [auto_monad_eliminator]: 0.00088704 [cse]: 0.00249775 [a_3]: 0.00595827 [Cycle 2]: 0.022768, [30] [expand_dump_flag]: 2.442e-05 [switch_simplify]: 0.00032019 [a_1]: 0.00904895 [recompute_prepare]: 6.06e-05 [updatestate_depend_eliminate]: 0.00054949 [updatestate_assign_eliminate]: 8.167e-05 [updatestate_loads_eliminate]: 0.00016488 [parameter_eliminate]: 3.42e-06 [a_2]: 0.00097846 [accelerated_algorithm]: 7.73e-05 [pynative_shard]: 5.727e-05 [auto_parallel]: 5.95e-06 [parallel]: 7.64e-06 [merge_comm]: 4.811e-05 [allreduce_fusion]: 3.208e-05 [virtual_dataset]: 5.507e-05 [get_grad_eliminate_]: 4.919e-05 [virtual_output]: 4.851e-05 [merge_forward]: 7.5e-05 [cell_reuse_recompute_pass]: 6.80004e-07 [cell_reuse_handle_not_recompute_node_pass]: 0.00013848 [meta_fg_expand]: 9.315e-05 [after_resolve]: 6.7e-05 [a_after_grad]: 6.242e-05 [renormalize]: 0.00882039 [real_op_eliminate]: 7.322e-05 [auto_monad_grad]: 4.92e-06 [auto_monad_eliminator]: 0.00027683 [cse]: 0.00087437 [a_3]: 0.0004267 [Cycle 3]: 0.00397917, [30] [expand_dump_flag]: 1.51e-06 [switch_simplify]: 5.205e-05 [a_1]: 0.00077609 [recompute_prepare]: 4.639e-05 [updatestate_depend_eliminate]: 0.00010422 [updatestate_assign_eliminate]: 7.176e-05 [updatestate_loads_eliminate]: 7.461e-05 [parameter_eliminate]: 2.37e-06 [a_2]: 0.00100678 [accelerated_algorithm]: 7.82e-05 [pynative_shard]: 5.799e-05 [auto_parallel]: 4.94e-06 [parallel]: 8.07e-06 [merge_comm]: 3.639e-05 [allreduce_fusion]: 2.511e-05 [virtual_dataset]: 5.477e-05 [get_grad_eliminate_]: 5.017e-05 [virtual_output]: 4.851e-05 [merge_forward]: 6.99e-05 [cell_reuse_recompute_pass]: 3.89999e-07 [cell_reuse_handle_not_recompute_node_pass]: 0.0001439 [meta_fg_expand]: 6.466e-05 [after_resolve]: 6.768e-05 [a_after_grad]: 6.416e-05 [renormalize]: 7.0002e-08 [real_op_eliminate]: 5.037e-05 [auto_monad_grad]: 2.47e-06 [auto_monad_eliminator]: 0.00016067 [cse]: 0.00027076 [a_3]: 0.00041488 [py_interpret_to_execute_after_opt_a]: 9.042e-05 [slice_cell_reuse_recomputed_activation]: 2e-06 [rewriter_after_opt_a]: 0.0019209 [convert_after_rewriter]: 0.00010015 [order_py_execute_after_rewriter]: 6.358e-05 [opt_b]: 0.0019392, [1] [Cycle 1]: 0.00193279, [7] [b_1]: 0.00130904 [b_2]: 5.763e-05 [updatestate_depend_eliminate]: 7.255e-05 [updatestate_assign_eliminate]: 6.879e-05 [updatestate_loads_eliminate]: 7.415e-05 [renormalize]: 4.047e-05 [cse]: 0.00026228 [cconv]: 6.077e-05 [opt_after_cconv]: 0.000735, [1] [Cycle 1]: 0.0007283, [7] [c_1]: 0.00018382 [parameter_eliminate]: 1.97e-06 [updatestate_depend_eliminate]: 8.129e-05 [updatestate_assign_eliminate]: 6.894e-05 [updatestate_loads_eliminate]: 7.062e-05 [cse]: 0.00024193 [renormalize]: 3.823e-05 [remove_dup_value]: 0.00029005 [tuple_transform]: 0.00053799, [1] [Cycle 1]: 0.00053176, [3] [d_1]: 0.00032378 [d_2]: 0.00014953 [renormalize]: 3.702e-05 [add_cache_embedding]: 0.00014758 [add_recomputation]: 0.00066198 [cse_after_recomputation]: 0.00022475, [1] [Cycle 1]: 0.00021825, [1] [cse]: 0.00020911 [environ_conv]: 8.686e-05 [label_micro_interleaved_index]: 1.87e-06 [label_fine_grained_interleaved_index]: 1.29001e-06 [assign_add_opt]: 4.029e-05 [slice_recompute_activation]: 1.56e-06 [micro_interleaved_order_control]: 1.1e-06 [full_micro_interleaved_order_control]: 1e-06 [comp_comm_scheduling]: 1.16e-06 [reorder_send_recv_between_fp_bp]: 1.05e-06 [comm_op_add_attrs]: 6.30003e-07 [add_comm_op_reuse_tag]: 1.914e-05 [overlap_opt_shard_in_pipeline]: 1.347e-05 [grouped_pairwise_exchange_alltoall]: 1.377e-05 [overlap_recompute_and_grad_model_parallel]: 1.23e-06 [overlap_grad_matmul_and_grad_allreduce]: 1.209e-05 [split_matmul_comm_elemetwise]: 1.34e-05 [split_layernorm_comm]: 1.13e-06 [process_send_recv_for_ge]: 2.32e-06 [handle_group_info]: 4.79995e-07 [auto_monad_reorder]: 0.00037208 [get_jit_bprop_graph]: 3.019e-05 [eliminate_special_op_node]: 0.00089423 [validate]: 0.00030164 [distribtued_split]: 0.00048807 [task_emit]: 0.09116 [execute]: 3.078e-05 Sums parse : 0.011352s : 1.48% symbol_resolve.resolve : 0.250952s : 32.63% graph_reusing : 0.000071s : 0.01% meta_unpack_prepare : 0.001189s : 0.15% pre_cconv : 0.000033s : 0.00% abstract_specialize : 0.242838s : 31.57% pack_expand : 0.000464s : 0.06% auto_monad : 0.003414s : 0.44% inline : 0.000033s : 0.00% pre_auto_parallel : 0.000038s : 0.00% pipeline_split : 0.000036s : 0.00% optimize.py_interpret_to_execute : 0.001121s : 0.15% optimize.rewriter_before_opt_a : 0.006048s : 0.79% optimize.opt_a.expand_dump_flag : 0.000094s : 0.01% optimize.opt_a.switch_simplify : 0.001435s : 0.19% optimize.opt_a.a_1 : 0.027766s : 3.61% optimize.opt_a.recompute_prepare : 0.000263s : 0.03% optimize.opt_a.updatestate_depend_eliminate : 0.001464s : 0.19% optimize.opt_a.updatestate_assign_eliminate : 0.000342s : 0.04% optimize.opt_a.updatestate_loads_eliminate : 0.000717s : 0.09% optimize.opt_a.parameter_eliminate : 0.000024s : 0.00% optimize.opt_a.a_2 : 0.004968s : 0.65% optimize.opt_a.accelerated_algorithm : 0.000251s : 0.03% optimize.opt_a.pynative_shard : 0.000164s : 0.02% optimize.opt_a.auto_parallel : 0.000016s : 0.00% optimize.opt_a.parallel : 0.000059s : 0.01% optimize.opt_a.merge_comm : 0.000164s : 0.02% optimize.opt_a.allreduce_fusion : 0.000103s : 0.01% optimize.opt_a.virtual_dataset : 0.000196s : 0.03% optimize.opt_a.get_grad_eliminate_ : 0.000170s : 0.02% optimize.opt_a.virtual_output : 0.000165s : 0.02% optimize.opt_a.merge_forward : 0.000265s : 0.03% optimize.opt_a.cell_reuse_recompute_pass : 0.000002s : 0.00% optimize.opt_a.cell_reuse_handle_not_recompute_node_pass : 0.000491s : 0.06% optimize.opt_a.meta_fg_expand : 0.033517s : 4.36% optimize.opt_a.after_resolve : 0.000570s : 0.07% optimize.opt_a.a_after_grad : 0.000767s : 0.10% optimize.opt_a.renormalize : 0.064419s : 8.38% optimize.opt_a.real_op_eliminate : 0.000798s : 0.10% optimize.opt_a.auto_monad_grad : 0.000486s : 0.06% optimize.opt_a.auto_monad_eliminator : 0.001325s : 0.17% optimize.opt_a.cse : 0.003643s : 0.47% optimize.opt_a.a_3 : 0.006800s : 0.88% optimize.py_interpret_to_execute_after_opt_a : 0.000090s : 0.01% optimize.slice_cell_reuse_recomputed_activation : 0.000002s : 0.00% optimize.rewriter_after_opt_a : 0.001921s : 0.25% optimize.convert_after_rewriter : 0.000100s : 0.01% optimize.order_py_execute_after_rewriter : 0.000064s : 0.01% optimize.opt_b.b_1 : 0.001309s : 0.17% optimize.opt_b.b_2 : 0.000058s : 0.01% optimize.opt_b.updatestate_depend_eliminate : 0.000073s : 0.01% optimize.opt_b.updatestate_assign_eliminate : 0.000069s : 0.01% optimize.opt_b.updatestate_loads_eliminate : 0.000074s : 0.01% optimize.opt_b.renormalize : 0.000040s : 0.01% optimize.opt_b.cse : 0.000262s : 0.03% optimize.cconv : 0.000061s : 0.01% optimize.opt_after_cconv.c_1 : 0.000184s : 0.02% optimize.opt_after_cconv.parameter_eliminate : 0.000002s : 0.00% optimize.opt_after_cconv.updatestate_depend_eliminate : 0.000081s : 0.01% optimize.opt_after_cconv.updatestate_assign_eliminate : 0.000069s : 0.01% optimize.opt_after_cconv.updatestate_loads_eliminate : 0.000071s : 0.01% optimize.opt_after_cconv.cse : 0.000242s : 0.03% optimize.opt_after_cconv.renormalize : 0.000038s : 0.00% optimize.remove_dup_value : 0.000290s : 0.04% optimize.tuple_transform.d_1 : 0.000324s : 0.04% optimize.tuple_transform.d_2 : 0.000150s : 0.02% optimize.tuple_transform.renormalize : 0.000037s : 0.00% optimize.add_cache_embedding : 0.000148s : 0.02% optimize.add_recomputation : 0.000662s : 0.09% optimize.cse_after_recomputation.cse : 0.000209s : 0.03% optimize.environ_conv : 0.000087s : 0.01% optimize.label_micro_interleaved_index : 0.000002s : 0.00% optimize.label_fine_grained_interleaved_index : 0.000001s : 0.00% optimize.assign_add_opt : 0.000040s : 0.01% optimize.slice_recompute_activation : 0.000002s : 0.00% optimize.micro_interleaved_order_control : 0.000001s : 0.00% optimize.full_micro_interleaved_order_control : 0.000001s : 0.00% optimize.comp_comm_scheduling : 0.000001s : 0.00% optimize.reorder_send_recv_between_fp_bp : 0.000001s : 0.00% optimize.comm_op_add_attrs : 0.000001s : 0.00% optimize.add_comm_op_reuse_tag : 0.000019s : 0.00% optimize.overlap_opt_shard_in_pipeline : 0.000013s : 0.00% optimize.grouped_pairwise_exchange_alltoall : 0.000014s : 0.00% optimize.overlap_recompute_and_grad_model_parallel : 0.000001s : 0.00% optimize.overlap_grad_matmul_and_grad_allreduce : 0.000012s : 0.00% optimize.split_matmul_comm_elemetwise : 0.000013s : 0.00% optimize.split_layernorm_comm : 0.000001s : 0.00% optimize.process_send_recv_for_ge : 0.000002s : 0.00% optimize.handle_group_info : 0.000000s : 0.00% auto_monad_reorder : 0.000372s : 0.05% get_jit_bprop_graph : 0.000030s : 0.00% eliminate_special_op_node : 0.000894s : 0.12% validate : 0.000302s : 0.04% distribtued_split : 0.000488s : 0.06% task_emit : 0.091160s : 11.85% execute : 0.000031s : 0.00% Time group info: ------[substitution.] 0.260218 4489 0.09% : 0.000225s : 57: substitution.arithmetic_simplify 0.02% : 0.000043s : 16: substitution.cast_eliminate 0.02% : 0.000047s : 55: substitution.depend_value_elim 0.01% : 0.000016s : 16: substitution.environ_get_add_eliminate 0.00% : 0.000006s : 8: substitution.environ_get_depend_swap 0.01% : 0.000021s : 32: substitution.environ_get_eliminate 0.02% : 0.000041s : 16: substitution.environ_get_set_eliminate 0.01% : 0.000037s : 94: substitution.float_depend_g_call 0.00% : 0.000008s : 16: substitution.float_environ_get_switch 0.00% : 0.000012s : 14: substitution.float_tuple_getitem_switch 94.71% : 0.246445s : 233: substitution.getattr_setattr_resolve 0.02% : 0.000053s : 120: substitution.graph_param_transform 0.00% : 0.000007s : 20: substitution.incorporate_call 0.00% : 0.000005s : 20: substitution.incorporate_call_switch 4.04% : 0.010521s : 377: substitution.inline 0.01% : 0.000014s : 34: substitution.less_batch_normalization 0.01% : 0.000031s : 160: substitution.load_eliminater 0.15% : 0.000386s : 505: substitution.meta_unpack_prepare 0.02% : 0.000054s : 72: substitution.minmaximum_grad 0.01% : 0.000018s : 8: substitution.partial_defer_inline 0.05% : 0.000127s : 94: substitution.partial_eliminate 0.00% : 0.000009s : 120: substitution.partial_unused_args_eliminate 0.02% : 0.000046s : 31: substitution.real_op_eliminate 0.01% : 0.000013s : 32: substitution.reduce_all_const_elim 0.01% : 0.000036s : 338: substitution.remove_not_recompute_node 0.13% : 0.000334s : 264: substitution.replace_applicator 0.01% : 0.000034s : 142: substitution.replace_old_param 0.00% : 0.000010s : 2: substitution.reshape_eliminate 0.00% : 0.000006s : 10: substitution.set_cell_output_no_recompute 0.00% : 0.000005s : 2: substitution.specialize_transform 0.01% : 0.000022s : 32: substitution.split_environ_get_set_with_tuple_value 0.01% : 0.000025s : 31: substitution.switch_simplify 0.07% : 0.000186s : 76: substitution.tuple_list_convert_item_index_to_positive 0.03% : 0.000085s : 92: substitution.tuple_list_get_item_const_eliminator 0.06% : 0.000166s : 92: substitution.tuple_list_get_item_depend_reorder 0.19% : 0.000493s : 283: substitution.tuple_list_get_item_eliminator 0.04% : 0.000117s : 92: substitution.tuple_list_get_set_item_eliminator 0.08% : 0.000213s : 416: substitution.updatestate_pure_node_eliminater 0.12% : 0.000300s : 467: substitution.updatestate_useless_node_eliminater ------[renormalize.] 0.064399 4 58.51% : 0.037680s : 2: renormalize.infer 41.49% : 0.026719s : 2: renormalize.specialize ------[replace.] 0.008318 886 0.07% : 0.000006s : 1: replace.arithmetic_simplify 1.00% : 0.000083s : 16: replace.cast_eliminate 0.64% : 0.000054s : 10: replace.depend_value_elim 0.96% : 0.000080s : 8: replace.environ_get_set_eliminate 36.19% : 0.003010s : 210: replace.getattr_setattr_resolve 30.97% : 0.002576s : 341: replace.inline 0.38% : 0.000032s : 1: replace.meta_unpack_prepare 4.89% : 0.000407s : 32: replace.partial_eliminate 1.80% : 0.000150s : 31: replace.real_op_eliminate 1.91% : 0.000159s : 9: replace.replace_applicator 3.44% : 0.000286s : 31: replace.switch_simplify 1.29% : 0.000108s : 16: replace.tuple_list_get_item_depend_reorder 16.20% : 0.001348s : 179: replace.tuple_list_get_item_eliminator 0.25% : 0.000021s : 1: replace.updatestate_useless_node_eliminater ------[match.] 0.256956 886 0.00% : 0.000011s : 1: match.arithmetic_simplify 0.02% : 0.000043s : 16: match.cast_eliminate 0.00% : 0.000008s : 10: match.depend_value_elim 0.01% : 0.000034s : 8: match.environ_get_set_eliminate 95.68% : 0.245856s : 210: match.getattr_setattr_resolve 4.00% : 0.010281s : 341: match.inline 0.05% : 0.000136s : 1: match.meta_unpack_prepare 0.04% : 0.000098s : 32: match.partial_eliminate 0.02% : 0.000046s : 31: match.real_op_eliminate 0.01% : 0.000033s : 9: match.replace_applicator 0.01% : 0.000025s : 31: match.switch_simplify 0.02% : 0.000059s : 16: match.tuple_list_get_item_depend_reorder 0.12% : 0.000320s : 179: match.tuple_list_get_item_eliminator 0.00% : 0.000006s : 1: match.updatestate_useless_node_eliminater ------[func_graph_cloner_run.] 0.042547 648 70.76% : 0.030104s : 267: func_graph_cloner_run.FuncGraphClonerGraph 29.24% : 0.012443s : 381: func_graph_cloner_run.FuncGraphSpecializer ------[meta_graph.] 0.000000 0 ------[manager.] 0.000000 0 ------[pynative] 0.000000 0 ------[others.] 0.299904 170 4.06% : 0.012168s : 103: opt.transform.opt_a 0.43% : 0.001293s : 23: opt.transform.opt_b 84.06% : 0.252106s : 6: opt.transform.opt_resolve 0.37% : 0.001108s : 1: opt.transforms.meta_unpack_prepare 10.78% : 0.032338s : 30: opt.transforms.opt_a 0.06% : 0.000182s : 1: opt.transforms.opt_after_cconv 0.02% : 0.000056s : 1: opt.transforms.opt_b 0.16% : 0.000470s : 2: opt.transforms.opt_trans_graph 0.06% : 0.000183s : 3: opt.transforms.special_op_eliminate [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:32.016.847 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1385] Run] End [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:32.016.877 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:846] SaveCompiledGraph] Save compiled func graph(381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316) phase(train.1704857851074634240.281471696508592.0)! [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:32.016.898 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:864] SaveCompiledGraph] End save compiled func graph! [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:32.016.914 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:942] CleanCompileRes] Clean compile resource start [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:32.027.519 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:956] CleanCompileRes] Clean compile resource end [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:32.027.567 [mindspore/ccsrc/pipeline/jit/ps/event_message_print.cc:37] PrintEventMessage] End compiling '_DataWrapper.construct'. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:32.027.583 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1039] CompileInner] Finish compiling. [WARNING] ME(164043:281473398948928,MainProcess):2024-01-10-11:37:32.286.61 [mindspore/parallel/_utils.py:259] You are suggested to use mindspore.context.set_auto_parallel_context(parameter_broadcast=True) or mindspore.common.set_seed() to share parameters among multi-devices. [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:37:32.031.043 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:37:32.031.114 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:37:32.031.151 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_1 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.031.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode false [INFO] PRE_ACT(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.031.523 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:237] SetMemAllocUintSize] Set mem alloc unit size, common 1073741824 persistent 1073741824 [INFO] DEVICE(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.031.543 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_pool.cc:124] AllocDeviceMem] Malloc Memory for Pool, size: 1073741824 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.031.744 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Float32, value=0.25), output index: 0 device address:0x2b10f2b0 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.031.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[1], dtype=Int64, value=[32]), output index: 0 device address:0x2ad41340 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.031.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode 1 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.032.016 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Int64, value=10), output index: 0 device address:0x2834f020 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.032.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode (32, 16, 5, 5) [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.032.174 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode (16, 6, 5, 5) [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.032.255 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[32, 1], dtype=Float32, value=[...]), output index: 0 device address:0x2b4c2450 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.032.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10]), output index: 0 device address:0x283e68e0 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.032.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode (32, 6, 14, 14) [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.032.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode true [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.032.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), output index: 0 device address:0x28346fb0 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.032.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode (6, 1, 5, 5) [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.032.756 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[1], dtype=Int32, value=[1]), output index: 0 device address:0x2ad13a60 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.032.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode -1 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.032.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode 0 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.032.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10]), output index: 0 device address:0x2acd94d0 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.033.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Float32, value=0), output index: 0 device address:0x2844f0c0 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.033.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Float32, value=1), output index: 0 device address:0x285972c0 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.033.236 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc3.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.033.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc3.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.033.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc2.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.033.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc2.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.033.644 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc1.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.033.793 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc1.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.033.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_conv2.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.034.018 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_conv1.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.034.118 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_global_step, device type:2 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.034.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc3.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.034.302 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_learning_rate, device type:2 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.034.392 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_momentum, device type:2 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.034.484 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc3.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.034.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc2.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.034.668 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc2.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.034.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc1.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.034.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc1.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.034.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.conv2.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.035.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.conv1.weight, device type:2 [INFO] PRE_ACT(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.035.188 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:237] SetMemAllocUintSize] Set mem alloc unit size, common 1073741824 persistent 1073741824 [INFO] DEVICE(164043,fffe3b74e0f0,python):2024-01-10-11:37:32.035.212 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_pool.cc:124] AllocDeviceMem] Malloc Memory for Pool, size: 1073741824 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:37:32.035.252 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] GE(164043,python):2024-01-10-11:37:32.113.395 [graph_var_manager.cc:1424][EVENT]167074 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:32.113.487 [graph_manager.cc:1248][EVENT]167074 PreRun:PreRun start: graph node size 2, session id 2, graph id 1, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:32.113.830 [atrace_api.c:28](tid:167074) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:32.113.861 [trace_rb_log.c:84](tid:167074) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:32.113.874 [atrace_api.c:32](tid:167074) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:32.113.891 [client_manager.cpp:157][SetProfilingCallback][tid:167074] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:32.114.371 [parallel_partitioner.cc:165][EVENT]167074 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.114.412 [parallel_partitioner.cc:178][EVENT]167074 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.114.456 [graph_prepare.cc:1378][EVENT]167074 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.114.668 [graph_manager.cc:1050][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [227] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.114.691 [graph_manager.cc:1052][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.114.759 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.114.787 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.114.847 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [48] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.114.860 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.114.904 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.114.918 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.114.930 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.115.024 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.115.045 [graph_manager.cc:1054][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [341] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.115.276 [graph_manager.cc:1055][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [194] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.116.089 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.116.119 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.116.130 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.116.141 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [243] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.116.153 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.116.162 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.116.171 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.116.179 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.116.188 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.117.492 [graph_manager.cc:1056][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2197] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.117.555 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.117.578 [graph_prepare.cc:1982][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [51] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.117.926 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.117.953 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.117.963 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.117.972 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [134] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.117.981 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.117.989 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.117.998 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.118.007 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [6] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.118.015 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.118.064 [graph_prepare.cc:1983][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [472] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.118.088 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.118.099 [graph_prepare.cc:1984][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.118.113 [graph_prepare.cc:1985][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.118.128 [graph_prepare.cc:1986][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.118.144 [graph_prepare.cc:1987][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.118.159 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.118.171 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.118.184 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.118.254 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.118.267 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.118.276 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.118.285 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.118.293 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.118.302 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.118.310 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.118.318 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.118.327 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.118.335 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.118.343 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.118.352 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.118.360 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.118.368 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.118.385 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.118.393 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.118.414 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.118.430 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.118.459 [graph_prepare.cc:1988][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [304] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.118.472 [graph_manager.cc:1065][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [944] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.130.312 [graph_manager.cc:1077][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11821] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.130.380 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.130.432 [graph_manager.cc:1080][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [82] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.031 [graph_manager.cc:1081][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [2579] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.074 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.089 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.101 [graph_manager.cc:1082][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.128 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.142 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.155 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.180 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.195 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.207 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.220 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.256 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.286 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.321 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [25] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.346 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.360 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.372 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.381 [graph_manager.cc:2700][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [258] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.463 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.133.477 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.133.486 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.133.495 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.133.503 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.133.512 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CastRemovePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.133.520 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.133.528 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.133.537 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.133.545 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.133.553 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [6] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.133.561 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.133.570 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [7] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.133.578 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.133.586 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.133.596 [graph_manager.cc:2741][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [197] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.605 [graph_manager.cc:2752][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.635 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.647 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.662 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.676 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.744 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.759 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.777 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.789 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.802 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.811 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.829 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.841 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.860 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.873 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.883 [graph_manager.cc:2810][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [251] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.908 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.133.920 [graph_manager.cc:2821][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [27] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.133.946 [graph_manager.cc:1087][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [830] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.134.077 [graph_manager.cc:1088][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [119] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.134.114 [graph_manager.cc:1089][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.134.136 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.134.150 [graph_manager.cc:1097][EVENT]167074 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:32.134.180 [graph_manager.cc:3325][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.134.286 [engine_place.cc:144][EVENT]167074 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.134.301 [engine_place.cc:144][EVENT]167074 Run:The time cost of aicpu_ascend_kernel::CheckSupported is [41] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.134.367 [graph_manager.cc:3351][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [173] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.134.387 [graph_manager.cc:3364][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.134.444 [engine_partitioner.cc:1139][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.134.463 [engine_partitioner.cc:1142][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.134.582 [engine_partitioner.cc:1148][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [109] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.134.610 [engine_partitioner.cc:1155][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.134.650 [engine_partitioner.cc:1164][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.134.683 [graph_manager.cc:3405][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [284] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.134.700 [graph_manager.cc:3412][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.136.676 [graph_manager.cc:3422][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [1961] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.136.711 [graph_manager.cc:3428][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.136.826 [graph_manager.cc:3467][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [93] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.136.846 [graph_manager.cc:3377][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [2447] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.136.864 [graph_manager.cc:1106][EVENT]167074 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [2691] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.136.879 [graph_manager.cc:1115][EVENT]167074 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:32.136.901 [graph_manager.cc:1130][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.136.932 [graph_manager.cc:1131][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.136.963 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.136.981 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.136.994 [graph_manager.cc:2837][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.055 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [8] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.137.067 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [0] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.137.077 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.137.087 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.137.099 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [8] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.137.108 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:37:32.137.121 [graph_manager.cc:2864][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [110] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.133 [graph_manager.cc:2872][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.150 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.164 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.179 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.191 [compile_nodes_pass.cc:88][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.205 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.216 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.300 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [71] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.327 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.344 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.357 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.370 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.389 [graph_manager.cc:2927][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [242] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.402 [graph_manager.cc:2937][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.428 [graph_manager.cc:2943][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.442 [graph_manager.cc:2950][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.597 [graph_manager.cc:2958][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.630 [graph_manager.cc:1132][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [676] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.798 [graph_manager.cc:1135][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [152] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.835 [graph_manager.cc:2975][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.937 [graph_manager.cc:2981][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [89] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.953 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.967 [graph_manager.cc:2986][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.137.980 [graph_manager.cc:1136][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [159] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.138.072 [graph_manager.cc:3555][EVENT]167074 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [67] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.138.124 [engine_partitioner.cc:1139][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.138.142 [engine_partitioner.cc:1142][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.138.223 [engine_partitioner.cc:1148][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [72] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.138.248 [engine_partitioner.cc:1155][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.138.280 [engine_partitioner.cc:1164][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.138.303 [graph_builder.cc:865][EVENT]167074 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [203] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.138.379 [graph_builder.cc:288][EVENT]167074 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [57] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.138.559 [graph_builder.cc:293][EVENT]167074 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [152] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.138.734 [model_builder.cc:1133][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [82] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.138.978 [block_mem_assigner.cc:4069][EVENT]167460 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164043,python):2024-01-10-11:37:32.138.981 [block_mem_assigner.cc:4069][EVENT]167461 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164043,python):2024-01-10-11:37:32.139.399 [graph_mem_assigner.cc:2166][EVENT]167074 SetInputOffset:[IMAS]AfterAssignMemory : online_1 memoffset[132096], memtype[2], theory_min[264192], zero_copy[132096], total_size[132096], no_reuse[132096], streams[1], topo_mode[DFS], mop[], io_reuse[0:0], alloc_mode[] [INFO] GE(164043,python):2024-01-10-11:37:32.139.488 [model_builder.cc:1144][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [730] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.139.514 [model_builder.cc:1152][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.139.533 [model_builder.cc:1157][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.139.638 [model_builder.cc:1167][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [93] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.139.659 [model_builder.cc:1174][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.139.682 [model_builder.cc:1180][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.139.714 [model_builder.cc:1184][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.139.735 [graph_builder.cc:304][EVENT]167074 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [1150] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:32.139.837 [logger.cc:1071] 167074 ModelBindStream: model_id=64, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:37:32.139.901 [task_generator.cc:804][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.139.959 [task_generator.cc:805][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [40] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.140.464 [task_generator.cc:814][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [488] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.140.483 [task_generator.cc:954][EVENT]167074 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [587] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.140.535 [task_generator.cc:967][EVENT]167074 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [30] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:32.140.558 [logger.cc:1084] 167074 ModelUnbindStream: model_id=64, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:37:32.140.613 [graph_builder.cc:310][EVENT]167074 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [865] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.140.713 [graph_manager.cc:1152][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2714] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.140.743 [graph_manager.cc:1164][EVENT]167074 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:32.140.775 [graph_manager.cc:1271][EVENT]167074 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [26487] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.140.789 [graph_manager.cc:1272][EVENT]167074 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:37:32.141.097 [atrace_api.c:93](tid:167074) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:37:32.141.115 [atrace_api.c:95](tid:167074) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:37:32.141.790 [model_introduction.cc:236][EVENT]167074 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(164043,python):2024-01-10-11:37:32.141.818 [model_introduction.cc:294][EVENT]167074 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(164043,python):2024-01-10-11:37:32.141.835 [model_introduction.cc:366][EVENT]167074 Data:model io_info size:114 [INFO] GE(164043,python):2024-01-10-11:37:32.145.252 [graph_converter.cc:838][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1300] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.145.439 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [139] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.145.924 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [456] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.146.016 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [60] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.146.036 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [81] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.146.081 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [31] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.146.115 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.146.148 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.146.219 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [57] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.146.284 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [49] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.146.298 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [63] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.146.330 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.146.359 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.146.376 [graph_converter.cc:849][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1082] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.146.578 [graph_converter.cc:853][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [190] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.147.194 [graph_converter.cc:857][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [597] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.147.310 [graph_converter.cc:862][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [89] micro second. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:37:32.151.663 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 101 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] GE(164043,python):2024-01-10-11:37:32.229.051 [graph_var_manager.cc:1424][EVENT]167073 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:32.229.149 [graph_manager.cc:1248][EVENT]167073 PreRun:PreRun start: graph node size 6, session id 3, graph id 2, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:32.230.081 [atrace_api.c:28](tid:167073) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:32.230.161 [trace_rb_log.c:84](tid:167073) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:32.230.178 [atrace_api.c:32](tid:167073) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:32.230.197 [client_manager.cpp:157][SetProfilingCallback][tid:167073] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:32.231.122 [parallel_partitioner.cc:165][EVENT]167073 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.231.168 [parallel_partitioner.cc:178][EVENT]167073 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.231.219 [graph_prepare.cc:1378][EVENT]167073 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.231.907 [graph_manager.cc:1050][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [706] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.231.940 [graph_manager.cc:1052][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.232.122 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.232.156 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.232.203 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.232.216 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.232.260 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.232.275 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.232.296 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.232.409 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.232.434 [graph_manager.cc:1054][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [481] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.232.648 [graph_manager.cc:1055][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [201] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.234.157 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:37:32.234.190 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [6] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.234.202 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.234.212 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferShapePass is [544] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.234.221 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [15] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.234.230 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:37:32.234.238 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [15] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.234.247 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.234.255 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.236.695 [graph_manager.cc:1056][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [4004] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.236.770 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.236.790 [graph_prepare.cc:1982][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [58] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.237.365 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:37:32.237.394 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.406 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.415 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferShapePass is [340] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.425 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.434 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:37:32.237.442 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.451 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.459 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.485 [graph_prepare.cc:1983][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [683] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.237.508 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.237.532 [graph_prepare.cc:1984][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.237.547 [graph_prepare.cc:1985][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.237.568 [graph_prepare.cc:1986][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.237.581 [graph_prepare.cc:1987][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.237.597 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.237.608 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.237.624 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.237.751 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.765 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.774 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.783 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.791 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DropOutPass is [4] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.800 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.808 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [4] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.817 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.825 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of StopGradientPass is [4] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.834 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.842 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [3] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.850 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.859 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.867 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [7] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.875 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.894 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.237.920 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.237.936 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.237.972 [graph_prepare.cc:1988][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [382] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.237.988 [graph_manager.cc:1065][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1256] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.250.830 [graph_manager.cc:1077][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12822] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.250.903 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.250.955 [graph_manager.cc:1080][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [86] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.258.878 [graph_manager.cc:1081][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [7904] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.258.923 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.258.939 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.258.951 [graph_manager.cc:1082][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.258.982 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.258.996 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.012 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.044 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.057 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.073 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.086 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.131 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.154 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.185 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.230 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.246 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.260 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.270 [graph_manager.cc:2700][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [294] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.415 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.259.429 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.259.439 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.259.448 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.259.457 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.259.465 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CastRemovePass is [12] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.259.474 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.259.482 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.259.490 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.259.499 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.259.507 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.259.515 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.259.524 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.259.532 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.259.540 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.259.550 [graph_manager.cc:2741][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [261] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.559 [graph_manager.cc:2752][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.580 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.599 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.619 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.635 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.647 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.659 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.678 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.692 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.707 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.718 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.732 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.743 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.763 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.777 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.786 [graph_manager.cc:2810][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [210] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.820 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.259.832 [graph_manager.cc:2821][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.860 [graph_manager.cc:1087][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [890] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.259.996 [graph_manager.cc:1088][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [124] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.260.038 [graph_manager.cc:1089][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.260.054 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.260.070 [graph_manager.cc:1097][EVENT]167073 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:32.260.091 [graph_manager.cc:3325][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.261.249 [engine_place.cc:144][EVENT]167073 Run:The time cost of AIcoreEngine::CheckSupported is [1032] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.261.291 [engine_place.cc:144][EVENT]167073 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.261.302 [engine_place.cc:144][EVENT]167073 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.261.397 [graph_manager.cc:3351][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [1292] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.261.415 [graph_manager.cc:3364][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.261.482 [engine_partitioner.cc:1139][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.261.501 [engine_partitioner.cc:1142][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.261.783 [engine_partitioner.cc:1148][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [271] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.261.837 [engine_partitioner.cc:1155][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.261.893 [engine_partitioner.cc:1164][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [41] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.261.933 [graph_manager.cc:3405][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [505] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.261.954 [graph_manager.cc:3412][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.272.411 [graph_manager.cc:3422][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [10442] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.272.456 [graph_manager.cc:3428][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.272.604 [graph_manager.cc:3467][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [126] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.272.626 [graph_manager.cc:3377][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [11199] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.272.643 [graph_manager.cc:1106][EVENT]167073 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [12558] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.272.656 [graph_manager.cc:1115][EVENT]167073 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:32.272.679 [graph_manager.cc:1130][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.272.712 [graph_manager.cc:1131][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.272.739 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.272.767 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.272.776 [graph_manager.cc:2837][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [46] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.272.868 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.272.884 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.272.894 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.272.902 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.272.911 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [4] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.272.920 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.272.929 [graph_manager.cc:2864][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [136] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.272.941 [graph_manager.cc:2872][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.272.961 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.272.975 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.272.991 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.004 [compile_nodes_pass.cc:88][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.015 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.025 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.118 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [83] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.146 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.160 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.174 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.187 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.206 [graph_manager.cc:2927][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [249] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.220 [graph_manager.cc:2937][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.235 [graph_manager.cc:2943][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.246 [graph_manager.cc:2950][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.419 [graph_manager.cc:2958][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [45] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.453 [graph_manager.cc:1132][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [724] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.521 [graph_manager.cc:1135][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [56] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.561 [graph_manager.cc:2975][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.591 [graph_manager.cc:2981][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.605 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.615 [graph_manager.cc:2986][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.624 [graph_manager.cc:1136][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [87] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.767 [graph_manager.cc:3555][EVENT]167073 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [107] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.875 [engine_partitioner.cc:1139][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.273.896 [engine_partitioner.cc:1142][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.274.075 [engine_partitioner.cc:1148][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [168] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.274.115 [engine_partitioner.cc:1155][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.274.157 [engine_partitioner.cc:1164][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.274.185 [graph_builder.cc:865][EVENT]167073 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [352] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:32.274.650 [logger.cc:1071] 167073 ModelBindStream: model_id=832, stream_id=1089, flag=0. [INFO] GE(164043,python):2024-01-10-11:37:32.274.686 [task_generator.cc:804][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [176] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.274.777 [task_generator.cc:805][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [62] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.275.576 [task_generator.cc:814][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [783] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.275.595 [task_generator.cc:954][EVENT]167073 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1086] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.275.656 [task_generator.cc:967][EVENT]167073 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [34] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:32.275.677 [logger.cc:1084] 167073 ModelUnbindStream: model_id=832, stream_id=1089, [INFO] GE(164043,python):2024-01-10-11:37:32.275.885 [graph_manager.cc:1152][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2232] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.275.910 [graph_manager.cc:1164][EVENT]167073 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:32.275.942 [graph_manager.cc:1271][EVENT]167073 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [44920] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.275.952 [graph_manager.cc:1272][EVENT]167073 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:37:32.276.262 [atrace_api.c:93](tid:167073) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:37:32.276.281 [atrace_api.c:95](tid:167073) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:37:32.282.853 [graph_converter.cc:838][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1975] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.283.024 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [123] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.283.673 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [621] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.283.941 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [240] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.283.967 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [267] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.284.223 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [244] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.284.272 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [27] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.284.309 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.284.561 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [239] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.284.664 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [82] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.284.681 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [100] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.284.716 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [25] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.284.747 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.284.778 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.284.896 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [98] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.284.983 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [74] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.284.996 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [87] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.285.027 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.285.056 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.285.070 [graph_converter.cc:849][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [2176] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.285.375 [graph_converter.cc:853][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [295] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.286.311 [graph_converter.cc:857][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [916] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.286.511 [graph_converter.cc:862][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [169] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.369.811 [graph_var_manager.cc:1424][EVENT]167075 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:32.369.912 [graph_manager.cc:1248][EVENT]167075 PreRun:PreRun start: graph node size 4, session id 4, graph id 3, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:32.370.802 [atrace_api.c:28](tid:167075) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:32.370.877 [trace_rb_log.c:84](tid:167075) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:32.370.892 [atrace_api.c:32](tid:167075) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:32.370.910 [client_manager.cpp:157][SetProfilingCallback][tid:167075] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:32.371.787 [parallel_partitioner.cc:165][EVENT]167075 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.371.829 [parallel_partitioner.cc:178][EVENT]167075 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.371.886 [graph_prepare.cc:1378][EVENT]167075 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.372.588 [graph_manager.cc:1050][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [724] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.372.620 [graph_manager.cc:1052][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.372.761 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.372.791 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.372.839 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.372.873 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.372.918 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.372.932 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.372.951 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.373.055 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.373.076 [graph_manager.cc:1054][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [442] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.373.294 [graph_manager.cc:1055][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [204] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.374.519 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:32.374.550 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.374.561 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.374.571 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [446] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.374.580 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.374.589 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:32.374.597 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [10] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.374.606 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.374.615 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.378.160 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:32.378.193 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.378.204 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.378.214 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [312] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.378.223 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.378.232 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:32.378.240 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.378.259 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.378.268 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.379.568 [graph_manager.cc:1056][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [6255] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.379.635 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.379.655 [graph_prepare.cc:1982][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [52] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.380.196 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:32.380.223 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.235 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.244 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [291] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.253 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.262 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:32.380.270 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.279 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.287 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.333 [graph_prepare.cc:1983][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [665] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.380.356 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.380.367 [graph_prepare.cc:1984][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.380.380 [graph_prepare.cc:1985][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.380.398 [graph_prepare.cc:1986][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.380.409 [graph_prepare.cc:1987][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.380.424 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.380.436 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.380.461 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.380.554 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of EnterPass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.567 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.576 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.585 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.593 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.602 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.610 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.618 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.626 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.635 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.643 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.651 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.659 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.667 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [7] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.676 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.684 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:32.380.706 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.380.719 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.380.750 [graph_prepare.cc:1988][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [332] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.380.763 [graph_manager.cc:1065][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1161] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.392.896 [graph_manager.cc:1077][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12113] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.392.990 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.393.051 [graph_manager.cc:1080][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [118] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.403.576 [graph_manager.cc:1081][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [10508] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.403.624 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.403.639 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.403.652 [graph_manager.cc:1082][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.403.685 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.403.701 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.403.716 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.403.882 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [155] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.403.901 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.009 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [97] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.026 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.077 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [40] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.099 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.118 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.217 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [88] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.236 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.249 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.259 [graph_manager.cc:2700][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [581] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.524 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.404.540 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AddNPass is [3] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.404.561 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.404.570 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.404.579 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.404.588 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CastRemovePass is [46] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.404.597 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.404.605 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [5] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.404.613 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [12] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.404.622 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.404.630 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [17] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.404.639 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [23] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.404.647 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [21] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.404.655 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [11] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.404.664 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [4] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.404.674 [graph_manager.cc:2741][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [396] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.683 [graph_manager.cc:2752][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.706 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.719 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.740 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.757 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.769 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.783 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.802 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.817 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.835 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.846 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.858 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.870 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.894 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.908 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.918 [graph_manager.cc:2810][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [216] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.404.966 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of IdentityPass is [7] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.404.978 [graph_manager.cc:2821][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [50] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.405.006 [graph_manager.cc:1087][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1334] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.405.588 [graph_manager.cc:1088][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [568] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.405.652 [graph_manager.cc:1089][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.405.676 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.405.703 [graph_manager.cc:1097][EVENT]167075 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:32.405.734 [graph_manager.cc:3325][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.415.556 [engine_place.cc:144][EVENT]167075 Run:The time cost of AIcoreEngine::CheckSupported is [9589] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.415.590 [engine_place.cc:144][EVENT]167075 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.415.604 [engine_place.cc:144][EVENT]167075 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.415.699 [graph_manager.cc:3351][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [9952] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.415.717 [graph_manager.cc:3364][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.415.801 [engine_partitioner.cc:1139][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [29] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.415.834 [engine_partitioner.cc:1142][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.416.046 [engine_partitioner.cc:1148][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [191] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.416.091 [engine_partitioner.cc:1155][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.416.142 [engine_partitioner.cc:1164][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.416.180 [graph_manager.cc:3405][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [450] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.416.198 [graph_manager.cc:3412][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.607.847 [graph_manager.cc:3422][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [191633] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.607.904 [graph_manager.cc:3428][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.094 [graph_manager.cc:3467][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [166] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.115 [graph_manager.cc:3377][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [192387] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.136 [graph_manager.cc:1106][EVENT]167075 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [202409] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.151 [graph_manager.cc:1115][EVENT]167075 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:32.608.177 [graph_manager.cc:1130][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.215 [graph_manager.cc:1131][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.248 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.271 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.281 [graph_manager.cc:2837][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [47] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.430 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [28] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.608.446 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.608.459 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.608.468 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of BitcastPass is [0] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.608.492 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [7] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.608.503 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [18] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:32.608.516 [graph_manager.cc:2864][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [216] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.530 [graph_manager.cc:2872][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.551 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.568 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.586 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.603 [compile_nodes_pass.cc:88][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.613 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.623 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.735 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [103] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.790 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [40] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.808 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.823 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.842 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.851 [graph_manager.cc:2927][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [306] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.864 [graph_manager.cc:2937][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.882 [graph_manager.cc:2943][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.608.896 [graph_manager.cc:2950][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.609.092 [graph_manager.cc:2958][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [59] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.609.129 [graph_manager.cc:1132][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [898] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.609.226 [graph_manager.cc:1135][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [74] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.609.272 [graph_manager.cc:2975][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.609.304 [graph_manager.cc:2981][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.609.321 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.609.331 [graph_manager.cc:2986][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.609.345 [graph_manager.cc:1136][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [101] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.609.764 [graph_manager.cc:3555][EVENT]167075 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [377] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.609.908 [engine_partitioner.cc:1139][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.609.943 [engine_partitioner.cc:1142][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.610.104 [engine_partitioner.cc:1148][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [149] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.610.145 [engine_partitioner.cc:1155][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.610.193 [engine_partitioner.cc:1164][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.610.231 [graph_builder.cc:865][EVENT]167075 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [390] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:32.610.779 [logger.cc:1071] 167075 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:37:32.610.815 [task_generator.cc:804][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [182] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.610.914 [task_generator.cc:805][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [86] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.612.877 [task_generator.cc:814][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1948] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.612.896 [task_generator.cc:954][EVENT]167075 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2263] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.612.970 [task_generator.cc:967][EVENT]167075 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [40] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:32.612.993 [logger.cc:1084] 167075 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:37:32.615.220 [graph_manager.cc:1152][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [5839] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.615.277 [graph_manager.cc:1164][EVENT]167075 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:32.615.317 [graph_manager.cc:1271][EVENT]167075 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [243618] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.615.331 [graph_manager.cc:1272][EVENT]167075 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:37:32.615.655 [atrace_api.c:93](tid:167075) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:37:32.615.674 [atrace_api.c:95](tid:167075) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:37:32.640.368 [graph_converter.cc:838][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [9893] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.640.597 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [172] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.642.366 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [1740] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.642.788 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [388] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.642.818 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [420] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.643.090 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [258] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.643.183 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [67] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.643.261 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [55] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.643.757 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [476] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.643.996 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [209] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.644.022 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [235] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.644.093 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [60] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.644.159 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [50] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.644.226 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [51] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.644.469 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [229] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.644.684 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [194] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.644.707 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [217] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.644.775 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [58] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.644.842 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [50] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.644.865 [graph_converter.cc:849][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4446] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.645.652 [graph_converter.cc:853][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [776] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.647.849 [graph_converter.cc:857][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2131] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.648.290 [graph_converter.cc:862][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [408] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.738.286 [graph_var_manager.cc:1424][EVENT]167073 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:32.738.384 [graph_manager.cc:1248][EVENT]167073 PreRun:PreRun start: graph node size 3, session id 5, graph id 4, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:32.738.683 [atrace_api.c:28](tid:167073) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:32.738.732 [trace_rb_log.c:84](tid:167073) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:32.738.745 [atrace_api.c:32](tid:167073) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:32.738.762 [client_manager.cpp:157][SetProfilingCallback][tid:167073] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:32.739.176 [parallel_partitioner.cc:165][EVENT]167073 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.739.213 [parallel_partitioner.cc:178][EVENT]167073 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.739.259 [graph_prepare.cc:1378][EVENT]167073 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.739.386 [graph_manager.cc:1050][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [144] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.739.411 [graph_manager.cc:1052][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.739.530 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.739.559 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.739.605 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.739.618 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.739.661 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.739.674 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.739.691 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.739.787 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.739.807 [graph_manager.cc:1054][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [383] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.740.041 [graph_manager.cc:1055][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [200] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.740.904 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.740.934 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.740.945 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.740.955 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferShapePass is [267] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.740.964 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.740.972 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.740.981 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.740.989 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.740.998 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.742.983 [graph_manager.cc:1056][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2922] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.743.050 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.069 [graph_prepare.cc:1982][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [50] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.743.432 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.743.457 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.469 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.479 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferShapePass is [179] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.488 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.496 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.743.505 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.513 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.522 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.563 [graph_prepare.cc:1983][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [478] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.743.598 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.743.611 [graph_prepare.cc:1984][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.743.625 [graph_prepare.cc:1985][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.743.640 [graph_prepare.cc:1986][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.743.652 [graph_prepare.cc:1987][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.743.667 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.743.679 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.743.693 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.743.775 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.787 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.796 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.805 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.814 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.822 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.831 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [0] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.839 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.847 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.856 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.864 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.873 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.881 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.889 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.905 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.914 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.743.936 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.743.948 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.743.977 [graph_prepare.cc:1988][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [315] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.743.991 [graph_manager.cc:1065][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [972] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.755.920 [graph_manager.cc:1077][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11908] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.755.991 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.756.037 [graph_manager.cc:1080][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [77] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.759.477 [graph_manager.cc:1081][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3422] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.759.521 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.759.535 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.759.546 [graph_manager.cc:1082][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.759.579 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.759.592 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.759.606 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.759.633 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.759.646 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.759.660 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.759.673 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.759.709 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [27] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.759.728 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.759.758 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.759.783 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.759.798 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.759.810 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.759.819 [graph_manager.cc:2700][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [245] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.759.924 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of EnterPass is [0] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.759.936 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AddNPass is [0] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.759.946 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.759.955 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.759.963 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.759.976 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CastRemovePass is [9] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.759.985 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.759.993 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.760.001 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.760.010 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.760.018 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.760.026 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.760.035 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.760.043 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.760.053 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.760.066 [graph_manager.cc:2741][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [228] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.078 [graph_manager.cc:2752][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.107 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.122 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.138 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.156 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.167 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.183 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.201 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.215 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.228 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.238 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.254 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.265 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.282 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.294 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.303 [graph_manager.cc:2810][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [199] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.332 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.760.346 [graph_manager.cc:2821][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.376 [graph_manager.cc:1087][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [810] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.509 [graph_manager.cc:1088][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [118] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.548 [graph_manager.cc:1089][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.565 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.580 [graph_manager.cc:1097][EVENT]167073 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:32.760.600 [graph_manager.cc:3325][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.958 [engine_place.cc:144][EVENT]167073 Run:The time cost of AIcoreEngine::CheckSupported is [253] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.987 [engine_place.cc:144][EVENT]167073 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.760.996 [engine_place.cc:144][EVENT]167073 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.761.066 [graph_manager.cc:3351][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [440] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.761.084 [graph_manager.cc:3364][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.761.146 [engine_partitioner.cc:1139][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.761.163 [engine_partitioner.cc:1142][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.761.293 [engine_partitioner.cc:1148][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [120] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.761.335 [engine_partitioner.cc:1155][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.761.383 [engine_partitioner.cc:1164][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.761.416 [graph_manager.cc:3405][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [320] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.761.433 [graph_manager.cc:3412][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.351 [graph_manager.cc:3422][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [8904] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.388 [graph_manager.cc:3428][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.506 [graph_manager.cc:3467][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [98] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.524 [graph_manager.cc:3377][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [9429] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.540 [graph_manager.cc:1106][EVENT]167073 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [9945] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.553 [graph_manager.cc:1115][EVENT]167073 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:32.770.575 [graph_manager.cc:1130][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.606 [graph_manager.cc:1131][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.639 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.656 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.666 [graph_manager.cc:2837][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.732 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.770.745 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.770.754 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.770.763 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.770.772 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [4] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.770.780 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.770.790 [graph_manager.cc:2864][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [108] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.802 [graph_manager.cc:2872][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.819 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.833 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.848 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.862 [compile_nodes_pass.cc:88][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.872 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.882 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.953 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [60] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.978 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.770.990 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.004 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.024 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.034 [graph_manager.cc:2927][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [218] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.046 [graph_manager.cc:2937][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.059 [graph_manager.cc:2943][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.070 [graph_manager.cc:2950][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.232 [graph_manager.cc:2958][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.262 [graph_manager.cc:1132][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [632] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.326 [graph_manager.cc:1135][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [52] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.356 [graph_manager.cc:2975][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.386 [graph_manager.cc:2981][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.400 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.410 [graph_manager.cc:2986][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.419 [graph_manager.cc:1136][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [77] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.528 [graph_manager.cc:3555][EVENT]167073 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [79] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.609 [engine_partitioner.cc:1139][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.624 [engine_partitioner.cc:1142][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.715 [engine_partitioner.cc:1148][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [81] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.744 [engine_partitioner.cc:1155][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.782 [engine_partitioner.cc:1164][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.771.802 [graph_builder.cc:865][EVENT]167073 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [222] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:32.772.071 [logger.cc:1071] 167073 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:37:32.772.112 [task_generator.cc:804][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [86] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.772.169 [task_generator.cc:805][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [44] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.772.827 [task_generator.cc:814][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [643] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.772.840 [task_generator.cc:954][EVENT]167073 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [816] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.772.897 [task_generator.cc:967][EVENT]167073 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [32] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:32.772.914 [logger.cc:1084] 167073 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:37:32.773.078 [graph_manager.cc:1152][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [1636] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.773.097 [graph_manager.cc:1164][EVENT]167073 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:32.773.128 [graph_manager.cc:1271][EVENT]167073 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [34033] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.773.140 [graph_manager.cc:1272][EVENT]167073 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:37:32.773.453 [atrace_api.c:93](tid:167073) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:37:32.773.471 [atrace_api.c:95](tid:167073) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:37:32.778.159 [graph_converter.cc:838][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1299] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.778.317 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [111] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.778.770 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [428] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.778.961 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [165] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.778.981 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [187] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.779.193 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [198] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.779.231 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.779.261 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.779.448 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [175] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.779.530 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [62] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.779.546 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [78] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.779.575 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.779.600 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.779.637 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.779.712 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [63] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.779.776 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [54] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.779.788 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [65] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.779.813 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.779.837 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.779.851 [graph_converter.cc:849][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1651] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.780.055 [graph_converter.cc:853][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [195] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.780.705 [graph_converter.cc:857][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [635] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.780.841 [graph_converter.cc:862][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [111] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.845.211 [graph_var_manager.cc:1424][EVENT]167074 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:32.845.306 [graph_manager.cc:1248][EVENT]167074 PreRun:PreRun start: graph node size 3, session id 6, graph id 5, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:32.845.587 [atrace_api.c:28](tid:167074) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:32.845.616 [trace_rb_log.c:84](tid:167074) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:32.845.629 [atrace_api.c:32](tid:167074) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:32.845.645 [client_manager.cpp:157][SetProfilingCallback][tid:167074] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:32.846.071 [parallel_partitioner.cc:165][EVENT]167074 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.846.109 [parallel_partitioner.cc:178][EVENT]167074 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.846.153 [graph_prepare.cc:1378][EVENT]167074 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.846.292 [graph_manager.cc:1050][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [154] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.846.316 [graph_manager.cc:1052][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.846.434 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.846.465 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.846.536 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.846.550 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.846.594 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.846.608 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.846.626 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.846.722 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.846.744 [graph_manager.cc:1054][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [416] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.846.950 [graph_manager.cc:1055][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [194] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.847.880 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.847.911 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.847.925 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.847.938 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [330] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.847.950 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.847.959 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.847.967 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [10] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.847.976 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.847.984 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.057 [graph_manager.cc:1056][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3088] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.850.125 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.144 [graph_prepare.cc:1982][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [49] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.850.537 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.850.564 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.586 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.596 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [194] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.605 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.614 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:32.850.622 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.631 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [8] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.639 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.680 [graph_prepare.cc:1983][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [522] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.850.703 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.850.715 [graph_prepare.cc:1984][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.850.729 [graph_prepare.cc:1985][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.850.744 [graph_prepare.cc:1986][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.850.756 [graph_prepare.cc:1987][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.850.771 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.850.783 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.850.796 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.850.876 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.888 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.897 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.906 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.915 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.923 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.932 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.951 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.960 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.968 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.977 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.985 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.850.994 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.851.002 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.851.010 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.851.019 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:32.851.041 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.851.053 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.851.082 [graph_prepare.cc:1988][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [317] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.851.095 [graph_manager.cc:1065][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1002] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.862.973 [graph_manager.cc:1077][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11858] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.863.043 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.863.090 [graph_manager.cc:1080][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [78] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.868.567 [graph_manager.cc:1081][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [5461] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.868.611 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.868.625 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.868.636 [graph_manager.cc:1082][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.868.667 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.868.682 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.868.708 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.868.842 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [123] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.868.860 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.868.936 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [67] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.868.951 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.868.995 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.016 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.034 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.111 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [66] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.127 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.139 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.149 [graph_manager.cc:2700][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [487] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.349 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.869.366 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.869.376 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.869.387 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.869.398 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.869.410 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CastRemovePass is [33] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.869.421 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.869.430 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.869.438 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [9] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.869.446 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.869.468 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.869.478 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.869.489 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.869.498 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [10] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.869.506 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.869.519 [graph_manager.cc:2741][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [350] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.528 [graph_manager.cc:2752][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.554 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.569 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.589 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.605 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.619 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.635 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.658 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.673 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.701 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.714 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.731 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.746 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.769 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.785 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.795 [graph_manager.cc:2810][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [245] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.833 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.869.852 [graph_manager.cc:2821][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [49] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.869.879 [graph_manager.cc:1087][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1225] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.870.358 [graph_manager.cc:1088][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [465] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.870.417 [graph_manager.cc:1089][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.870.438 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.870.456 [graph_manager.cc:1097][EVENT]167074 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:32.870.477 [graph_manager.cc:3325][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.877.898 [engine_place.cc:144][EVENT]167074 Run:The time cost of AIcoreEngine::CheckSupported is [7240] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.877.932 [engine_place.cc:144][EVENT]167074 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.877.942 [engine_place.cc:144][EVENT]167074 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.878.028 [graph_manager.cc:3351][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [7536] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.878.047 [graph_manager.cc:3364][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.878.115 [engine_partitioner.cc:1139][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.878.143 [engine_partitioner.cc:1142][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.878.281 [engine_partitioner.cc:1148][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [127] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.878.320 [engine_partitioner.cc:1155][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [27] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.878.367 [engine_partitioner.cc:1164][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.878.401 [graph_manager.cc:3405][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [340] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.878.420 [graph_manager.cc:3412][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.090 [graph_manager.cc:3422][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [72656] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.162 [graph_manager.cc:3428][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.337 [graph_manager.cc:3467][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [152] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.356 [graph_manager.cc:3377][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [73296] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.373 [graph_manager.cc:1106][EVENT]167074 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [80904] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.387 [graph_manager.cc:1115][EVENT]167074 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:32.951.414 [graph_manager.cc:1130][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.448 [graph_manager.cc:1131][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.477 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.496 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.507 [graph_manager.cc:2837][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [42] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.634 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [24] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.951.646 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.951.655 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.951.664 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.951.673 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [8] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.951.681 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [14] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:37:32.951.691 [graph_manager.cc:2864][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [165] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.702 [graph_manager.cc:2872][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.722 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.737 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.754 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.768 [compile_nodes_pass.cc:88][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.786 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.797 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.895 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [88] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.937 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.950 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.965 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.978 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.951.987 [graph_manager.cc:2927][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [269] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.952.001 [graph_manager.cc:2937][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.952.016 [graph_manager.cc:2943][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.952.027 [graph_manager.cc:2950][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.952.257 [graph_manager.cc:2958][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [48] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.952.289 [graph_manager.cc:1132][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [828] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.952.376 [graph_manager.cc:1135][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [74] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.952.412 [graph_manager.cc:2975][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.952.443 [graph_manager.cc:2981][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.952.457 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.952.467 [graph_manager.cc:2986][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.952.476 [graph_manager.cc:1136][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [83] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.952.760 [graph_manager.cc:3555][EVENT]167074 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [245] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.952.893 [engine_partitioner.cc:1139][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [29] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.952.919 [engine_partitioner.cc:1142][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.953.037 [engine_partitioner.cc:1148][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [107] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.953.071 [engine_partitioner.cc:1155][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.953.114 [engine_partitioner.cc:1164][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.953.140 [graph_builder.cc:865][EVENT]167074 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [302] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:32.953.507 [logger.cc:1071] 167074 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:37:32.953.541 [task_generator.cc:804][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [88] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.953.619 [task_generator.cc:805][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [63] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.954.980 [task_generator.cc:814][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1345] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.954.997 [task_generator.cc:954][EVENT]167074 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1544] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.955.061 [task_generator.cc:967][EVENT]167074 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [34] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:32.955.084 [logger.cc:1084] 167074 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:37:32.955.805 [graph_manager.cc:1152][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3298] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.955.840 [graph_manager.cc:1164][EVENT]167074 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:32.955.881 [graph_manager.cc:1271][EVENT]167074 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [109893] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.955.893 [graph_manager.cc:1272][EVENT]167074 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:37:32.956.214 [atrace_api.c:93](tid:167074) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:37:32.956.232 [atrace_api.c:95](tid:167074) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:37:32.974.058 [graph_converter.cc:838][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [6455] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.974.255 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [146] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.975.407 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [1124] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.975.691 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [253] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.975.718 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [281] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.975.968 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [222] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.976.034 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [43] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.976.089 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.976.435 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [330] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.976.600 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [138] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.976.620 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [159] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.976.670 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [39] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.976.717 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.976.767 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.976.939 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [158] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.977.082 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [128] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.977.097 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [143] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.977.143 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.977.187 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.977.204 [graph_converter.cc:849][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [3101] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.977.753 [graph_converter.cc:853][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [538] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.979.220 [graph_converter.cc:857][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1441] micro second. [INFO] GE(164043,python):2024-01-10-11:37:32.979.524 [graph_converter.cc:862][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [272] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.057.124 [graph_var_manager.cc:1424][EVENT]167075 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:33.057.224 [graph_manager.cc:1248][EVENT]167075 PreRun:PreRun start: graph node size 4, session id 7, graph id 6, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:33.057.486 [atrace_api.c:28](tid:167075) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:33.057.512 [trace_rb_log.c:84](tid:167075) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:33.057.524 [atrace_api.c:32](tid:167075) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:33.057.541 [client_manager.cpp:157][SetProfilingCallback][tid:167075] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:33.057.977 [parallel_partitioner.cc:165][EVENT]167075 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.058.018 [parallel_partitioner.cc:178][EVENT]167075 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.058.063 [graph_prepare.cc:1378][EVENT]167075 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.058.199 [graph_manager.cc:1050][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [153] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.058.223 [graph_manager.cc:1052][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.058.365 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.058.395 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.058.443 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.058.456 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.058.500 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.058.514 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.058.532 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.058.628 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.058.648 [graph_manager.cc:1054][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [413] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.058.866 [graph_manager.cc:1055][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [204] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.060.055 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:33.060.085 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.060.097 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.060.107 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [475] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.060.116 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.060.125 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:33.060.133 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.060.154 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.060.164 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.063.041 [graph_manager.cc:1056][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [4154] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.063.111 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.063.129 [graph_prepare.cc:1982][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [52] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.063.691 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:33.063.719 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.063.730 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.063.740 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [351] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.063.749 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.063.757 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:33.063.766 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.063.774 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.063.783 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.063.809 [graph_prepare.cc:1983][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [666] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.063.833 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.063.844 [graph_prepare.cc:1984][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.063.858 [graph_prepare.cc:1985][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.063.874 [graph_prepare.cc:1986][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.063.885 [graph_prepare.cc:1987][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.063.900 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.063.912 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.063.937 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.064.032 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.064.045 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.064.054 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.064.062 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.064.071 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.064.079 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.064.088 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.064.096 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.064.105 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.064.113 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.064.121 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.064.130 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.064.138 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.064.146 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.064.154 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.064.163 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.064.185 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.064.197 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.064.230 [graph_prepare.cc:1988][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [335] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.064.243 [graph_manager.cc:1065][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1166] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.077.376 [graph_manager.cc:1077][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13113] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.077.475 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.077.538 [graph_manager.cc:1080][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [123] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.088.710 [graph_manager.cc:1081][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [11154] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.088.756 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.088.772 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.088.784 [graph_manager.cc:1082][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.088.817 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.088.833 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.088.849 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.088.998 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [139] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.016 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.109 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [81] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.125 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.174 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [39] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.196 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.216 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.304 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [78] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.322 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.337 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.347 [graph_manager.cc:2700][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [536] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.589 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.089.604 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.089.615 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.089.634 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.089.643 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.089.652 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CastRemovePass is [40] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.089.661 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [5] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.089.670 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.089.678 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [11] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.089.710 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.089.720 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.089.729 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.089.737 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.089.745 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [9] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.089.754 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [6] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.089.763 [graph_manager.cc:2741][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [398] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.774 [graph_manager.cc:2752][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.799 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.812 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.834 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.850 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.862 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.875 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.895 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.910 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.931 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.942 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.955 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.967 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.089.990 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.090.003 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.090.013 [graph_manager.cc:2810][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [218] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.090.058 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.090.070 [graph_manager.cc:2821][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [48] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.090.097 [graph_manager.cc:1087][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1294] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.090.667 [graph_manager.cc:1088][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [555] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.090.734 [graph_manager.cc:1089][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.090.759 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.090.777 [graph_manager.cc:1097][EVENT]167075 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:33.090.799 [graph_manager.cc:3325][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.100.553 [engine_place.cc:144][EVENT]167075 Run:The time cost of AIcoreEngine::CheckSupported is [9538] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.100.587 [engine_place.cc:144][EVENT]167075 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.100.598 [engine_place.cc:144][EVENT]167075 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.100.693 [graph_manager.cc:3351][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [9880] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.100.711 [graph_manager.cc:3364][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.100.789 [engine_partitioner.cc:1139][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [28] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.100.819 [engine_partitioner.cc:1142][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.101.002 [engine_partitioner.cc:1148][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [162] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.101.043 [engine_partitioner.cc:1155][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [28] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.101.092 [engine_partitioner.cc:1164][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.101.128 [graph_manager.cc:3405][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [404] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.101.148 [graph_manager.cc:3412][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.226.117 [graph_manager.cc:3422][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [124955] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.226.174 [graph_manager.cc:3428][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.226.352 [graph_manager.cc:3467][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [154] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.226.376 [graph_manager.cc:3377][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [125652] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.226.394 [graph_manager.cc:1106][EVENT]167075 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [135602] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.226.408 [graph_manager.cc:1115][EVENT]167075 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:33.226.438 [graph_manager.cc:1130][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.226.476 [graph_manager.cc:1131][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.226.511 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.226.536 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.226.550 [graph_manager.cc:2837][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [53] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.226.689 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [27] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.226.707 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.226.717 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.226.727 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.226.736 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [8] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.226.760 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [14] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.226.772 [graph_manager.cc:2864][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [203] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.226.785 [graph_manager.cc:2872][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.226.806 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.226.821 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.226.839 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.226.853 [compile_nodes_pass.cc:88][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.226.863 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.226.873 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.226.984 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [102] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.227.039 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.227.057 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.227.077 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.227.094 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.227.104 [graph_manager.cc:2927][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [303] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.227.121 [graph_manager.cc:2937][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.227.140 [graph_manager.cc:2943][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.227.151 [graph_manager.cc:2950][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.227.351 [graph_manager.cc:2958][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [58] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.227.389 [graph_manager.cc:1132][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [895] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.227.472 [graph_manager.cc:1135][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [66] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.227.533 [graph_manager.cc:2975][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.227.565 [graph_manager.cc:2981][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.227.581 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.227.591 [graph_manager.cc:2986][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.227.600 [graph_manager.cc:1136][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [89] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.227.935 [graph_manager.cc:3555][EVENT]167075 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [293] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.228.065 [engine_partitioner.cc:1139][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [31] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.228.099 [engine_partitioner.cc:1142][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.228.251 [engine_partitioner.cc:1148][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [138] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.228.287 [engine_partitioner.cc:1155][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.228.330 [engine_partitioner.cc:1164][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.228.358 [graph_builder.cc:865][EVENT]167075 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [355] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:33.228.752 [logger.cc:1071] 167075 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:37:33.228.785 [task_generator.cc:804][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [89] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.228.872 [task_generator.cc:805][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [74] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.230.638 [task_generator.cc:814][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1750] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.230.655 [task_generator.cc:954][EVENT]167075 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1960] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.230.728 [task_generator.cc:967][EVENT]167075 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [39] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:33.230.754 [logger.cc:1084] 167075 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:37:33.232.036 [graph_manager.cc:1152][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4401] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.232.075 [graph_manager.cc:1164][EVENT]167075 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:33.232.127 [graph_manager.cc:1271][EVENT]167075 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [174240] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.232.139 [graph_manager.cc:1272][EVENT]167075 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:37:33.232.462 [atrace_api.c:93](tid:167075) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:37:33.232.480 [atrace_api.c:95](tid:167075) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:37:33.262.426 [graph_converter.cc:838][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [11447] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.262.647 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [164] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.264.156 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [1480] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.264.551 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [363] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.264.579 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [393] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.264.840 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [247] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.264.925 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [60] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.264.994 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [51] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.265.456 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [446] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.265.674 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [191] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.265.708 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [226] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.265.776 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [57] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.265.836 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [45] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.265.896 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [47] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.266.122 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [214] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.266.315 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [175] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.266.334 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [194] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.266.395 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [51] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.266.453 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [44] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.266.472 [graph_converter.cc:849][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [3995] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.267.182 [graph_converter.cc:853][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [698] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.269.145 [graph_converter.cc:857][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1922] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.269.542 [graph_converter.cc:862][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [365] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.344.314 [graph_var_manager.cc:1424][EVENT]167073 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:33.344.416 [graph_manager.cc:1248][EVENT]167073 PreRun:PreRun start: graph node size 4, session id 8, graph id 7, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:33.344.682 [atrace_api.c:28](tid:167073) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:33.344.713 [trace_rb_log.c:84](tid:167073) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:33.344.726 [atrace_api.c:32](tid:167073) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:33.344.743 [client_manager.cpp:157][SetProfilingCallback][tid:167073] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:33.345.116 [parallel_partitioner.cc:165][EVENT]167073 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.345.153 [parallel_partitioner.cc:178][EVENT]167073 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.345.198 [graph_prepare.cc:1378][EVENT]167073 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.345.332 [graph_manager.cc:1050][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [150] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.345.356 [graph_manager.cc:1052][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.345.489 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.345.519 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.345.567 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.345.580 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.345.624 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.345.637 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.345.654 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.345.780 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.345.801 [graph_manager.cc:1054][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [432] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.346.030 [graph_manager.cc:1055][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [191] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.347.034 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:33.347.064 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.347.076 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.347.086 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferShapePass is [330] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.347.095 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.347.104 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:33.347.113 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.347.122 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.347.130 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferValuePass is [7] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.349.207 [graph_manager.cc:1056][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3156] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.349.274 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.349.293 [graph_prepare.cc:1982][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [51] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.349.794 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:33.349.823 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.349.834 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.349.844 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferShapePass is [237] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.349.853 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [42] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.349.861 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:33.349.870 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.349.879 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.349.888 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.349.945 [graph_prepare.cc:1983][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [638] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.349.981 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.349.993 [graph_prepare.cc:1984][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.350.008 [graph_prepare.cc:1985][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.350.022 [graph_prepare.cc:1986][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.350.033 [graph_prepare.cc:1987][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.350.048 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.350.059 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.350.073 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.350.164 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.350.178 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.350.187 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.350.196 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.350.204 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.350.213 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.350.221 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.350.230 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.350.239 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.350.248 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.350.256 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.350.265 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.350.273 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.350.281 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.350.289 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.350.305 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.350.328 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.350.341 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.350.374 [graph_prepare.cc:1988][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [331] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.350.386 [graph_manager.cc:1065][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1146] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.362.660 [graph_manager.cc:1077][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12253] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.362.797 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.362.851 [graph_manager.cc:1080][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [152] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.369.463 [graph_manager.cc:1081][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [6594] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.369.509 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.369.526 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.369.538 [graph_manager.cc:1082][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.369.568 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.369.582 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.369.596 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.369.625 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.369.639 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.369.654 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.369.668 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.369.756 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [79] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.369.777 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.369.813 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.369.842 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.369.858 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.369.870 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.369.879 [graph_manager.cc:2700][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [317] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.000 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of EnterPass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.370.015 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.370.025 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.370.034 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.370.043 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.370.051 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.370.060 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.370.069 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.370.077 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.370.086 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.370.094 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.370.102 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.370.111 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.370.119 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.370.127 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.370.137 [graph_manager.cc:2741][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [239] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.146 [graph_manager.cc:2752][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.168 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.188 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.205 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.220 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.232 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.244 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.263 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.276 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.289 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.298 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.311 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.321 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.340 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.353 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.362 [graph_manager.cc:2810][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [198] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.392 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.370.405 [graph_manager.cc:2821][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.432 [graph_manager.cc:1087][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [876] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.567 [graph_manager.cc:1088][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [122] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.606 [graph_manager.cc:1089][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.623 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.370.638 [graph_manager.cc:1097][EVENT]167073 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:33.370.658 [graph_manager.cc:3325][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.371.043 [engine_place.cc:144][EVENT]167073 Run:The time cost of AIcoreEngine::CheckSupported is [279] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.371.071 [engine_place.cc:144][EVENT]167073 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.371.080 [engine_place.cc:144][EVENT]167073 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.371.157 [graph_manager.cc:3351][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [476] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.371.176 [graph_manager.cc:3364][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.371.240 [engine_partitioner.cc:1139][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.371.258 [engine_partitioner.cc:1142][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.371.413 [engine_partitioner.cc:1148][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [145] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.371.457 [engine_partitioner.cc:1155][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.371.504 [engine_partitioner.cc:1164][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.371.542 [graph_manager.cc:3405][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [354] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.371.562 [graph_manager.cc:3412][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.098 [graph_manager.cc:3422][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [41518] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.149 [graph_manager.cc:3428][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.296 [graph_manager.cc:3467][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [125] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.316 [graph_manager.cc:3377][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [42129] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.331 [graph_manager.cc:1106][EVENT]167073 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [42680] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.344 [graph_manager.cc:1115][EVENT]167073 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:33.413.368 [graph_manager.cc:1130][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.400 [graph_manager.cc:1131][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.443 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.460 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.470 [graph_manager.cc:2837][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.558 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.413.571 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.413.580 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.413.589 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.413.598 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.413.607 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.413.616 [graph_manager.cc:2864][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [128] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.627 [graph_manager.cc:2872][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.647 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.661 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.677 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.741 [compile_nodes_pass.cc:88][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.753 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.763 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.842 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [69] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.870 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.883 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.896 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.909 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.927 [graph_manager.cc:2927][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [283] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.941 [graph_manager.cc:2937][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.954 [graph_manager.cc:2943][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.413.965 [graph_manager.cc:2950][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.414.137 [graph_manager.cc:2958][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.414.170 [graph_manager.cc:1132][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [739] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.414.241 [graph_manager.cc:1135][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [59] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.414.272 [graph_manager.cc:2975][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.414.301 [graph_manager.cc:2981][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.414.315 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.414.325 [graph_manager.cc:2986][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.414.334 [graph_manager.cc:1136][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [76] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.414.456 [graph_manager.cc:3555][EVENT]167073 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [90] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.414.547 [engine_partitioner.cc:1139][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.414.563 [engine_partitioner.cc:1142][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.414.688 [engine_partitioner.cc:1148][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [115] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.414.721 [engine_partitioner.cc:1155][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.414.761 [engine_partitioner.cc:1164][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.414.785 [graph_builder.cc:865][EVENT]167073 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [273] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:33.415.079 [logger.cc:1071] 167073 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:37:33.415.123 [task_generator.cc:804][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [95] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.415.185 [task_generator.cc:805][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [48] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.415.880 [task_generator.cc:814][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [680] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.415.894 [task_generator.cc:954][EVENT]167073 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [867] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.415.951 [task_generator.cc:967][EVENT]167073 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [31] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:33.415.968 [logger.cc:1084] 167073 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:37:33.416.499 [graph_manager.cc:1152][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2139] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.416.533 [graph_manager.cc:1164][EVENT]167073 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:33.416.568 [graph_manager.cc:1271][EVENT]167073 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [71537] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.416.580 [graph_manager.cc:1272][EVENT]167073 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:37:33.416.900 [atrace_api.c:93](tid:167073) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:37:33.416.917 [atrace_api.c:95](tid:167073) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:37:33.429.096 [graph_converter.cc:838][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [3783] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.429.264 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [119] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.429.813 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [523] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.430.036 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [194] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.430.060 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [219] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.430.278 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [206] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.430.318 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.430.350 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.430.548 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [185] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.430.633 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [67] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.430.646 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [81] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.430.675 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.430.700 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.430.740 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.430.820 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [69] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.430.889 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [58] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.430.901 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [69] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.430.927 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.430.950 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.430.964 [graph_converter.cc:849][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1825] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.431.189 [graph_converter.cc:853][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [215] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.431.932 [graph_converter.cc:857][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [726] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.432.082 [graph_converter.cc:862][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [123] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.500.360 [graph_var_manager.cc:1424][EVENT]167074 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:33.500.460 [graph_manager.cc:1248][EVENT]167074 PreRun:PreRun start: graph node size 3, session id 9, graph id 8, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:33.500.711 [atrace_api.c:28](tid:167074) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:33.500.738 [trace_rb_log.c:84](tid:167074) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:33.500.751 [atrace_api.c:32](tid:167074) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:33.500.768 [client_manager.cpp:157][SetProfilingCallback][tid:167074] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:33.501.128 [parallel_partitioner.cc:165][EVENT]167074 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.501.164 [parallel_partitioner.cc:178][EVENT]167074 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.501.208 [graph_prepare.cc:1378][EVENT]167074 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.501.373 [graph_manager.cc:1050][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [180] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.501.396 [graph_manager.cc:1052][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.501.516 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.501.547 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.501.625 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.501.639 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.501.684 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.501.736 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.501.754 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.501.853 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.501.874 [graph_manager.cc:1054][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [465] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.502.084 [graph_manager.cc:1055][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [196] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.502.917 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:33.502.947 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.502.959 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.502.969 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [248] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.502.978 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.502.987 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:33.502.995 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [10] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.503.003 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.503.012 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.504.949 [graph_manager.cc:1056][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2846] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.505.016 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.035 [graph_prepare.cc:1982][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [50] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.505.371 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:33.505.396 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.418 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.428 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [174] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.437 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.445 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:33.505.454 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.462 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.471 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.496 [graph_prepare.cc:1983][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [447] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.505.519 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.505.530 [graph_prepare.cc:1984][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.505.543 [graph_prepare.cc:1985][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.505.558 [graph_prepare.cc:1986][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.505.569 [graph_prepare.cc:1987][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.505.583 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.505.597 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.505.611 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.505.729 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.743 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.752 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.761 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.769 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.778 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.786 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.804 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.813 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of StopGradientPass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.822 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.830 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.839 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SnapshotPass is [40] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.847 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.855 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.863 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.872 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.505.894 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.505.908 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.505.939 [graph_prepare.cc:1988][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [360] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.505.952 [graph_manager.cc:1065][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [968] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.517.850 [graph_manager.cc:1077][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11878] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.517.917 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.517.963 [graph_manager.cc:1080][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [75] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.521.412 [graph_manager.cc:1081][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3432] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.521.456 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.521.472 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.521.483 [graph_manager.cc:1082][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.521.515 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.521.531 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.521.555 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.521.583 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.521.598 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.521.612 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.521.626 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.521.665 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [28] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.521.683 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.521.722 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.521.749 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.521.764 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.521.776 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.521.785 [graph_manager.cc:2700][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [276] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.521.891 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of EnterPass is [0] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.521.904 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.521.913 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.521.922 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.521.931 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.521.939 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CastRemovePass is [8] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.521.948 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.521.956 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.521.964 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.521.973 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.521.989 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [6] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.521.998 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.522.007 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.522.015 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.522.023 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.522.033 [graph_manager.cc:2741][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [230] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.042 [graph_manager.cc:2752][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.065 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.076 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.093 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.108 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.120 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.132 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.151 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.165 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.179 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.189 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.202 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.214 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.232 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.245 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.255 [graph_manager.cc:2810][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [194] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.281 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.522.301 [graph_manager.cc:2821][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.329 [graph_manager.cc:1087][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [827] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.462 [graph_manager.cc:1088][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [121] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.501 [graph_manager.cc:1089][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.520 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.534 [graph_manager.cc:1097][EVENT]167074 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:33.522.554 [graph_manager.cc:3325][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.921 [engine_place.cc:144][EVENT]167074 Run:The time cost of AIcoreEngine::CheckSupported is [275] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.948 [engine_place.cc:144][EVENT]167074 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.522.958 [engine_place.cc:144][EVENT]167074 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.523.029 [graph_manager.cc:3351][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [461] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.523.047 [graph_manager.cc:3364][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.523.113 [engine_partitioner.cc:1139][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.523.131 [engine_partitioner.cc:1142][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.523.261 [engine_partitioner.cc:1148][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [120] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.523.302 [engine_partitioner.cc:1155][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [27] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.523.349 [engine_partitioner.cc:1164][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.523.381 [graph_manager.cc:3405][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [321] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.523.399 [graph_manager.cc:3412][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.531.502 [graph_manager.cc:3422][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [8087] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.531.547 [graph_manager.cc:3428][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.531.697 [graph_manager.cc:3467][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [116] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.531.718 [graph_manager.cc:3377][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [8659] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.531.735 [graph_manager.cc:1106][EVENT]167074 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [9187] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.531.748 [graph_manager.cc:1115][EVENT]167074 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:33.531.773 [graph_manager.cc:1130][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.531.807 [graph_manager.cc:1131][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.531.832 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.531.850 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.531.859 [graph_manager.cc:2837][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.531.934 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.531.947 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.531.958 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondRemovePass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.531.968 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of BitcastPass is [0] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.531.977 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.531.986 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:33.531.995 [graph_manager.cc:2864][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [120] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.008 [graph_manager.cc:2872][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.028 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.042 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.065 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.079 [compile_nodes_pass.cc:88][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.098 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.109 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.184 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [64] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.214 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.227 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.239 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.252 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.261 [graph_manager.cc:2927][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [237] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.274 [graph_manager.cc:2937][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.287 [graph_manager.cc:2943][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.298 [graph_manager.cc:2950][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.464 [graph_manager.cc:2958][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.496 [graph_manager.cc:1132][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [675] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.569 [graph_manager.cc:1135][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [59] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.601 [graph_manager.cc:2975][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.634 [graph_manager.cc:2981][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.649 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.659 [graph_manager.cc:2986][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.668 [graph_manager.cc:1136][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [82] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.788 [graph_manager.cc:3555][EVENT]167074 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [86] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.872 [engine_partitioner.cc:1139][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.897 [engine_partitioner.cc:1142][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.532.992 [engine_partitioner.cc:1148][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [84] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.533.020 [engine_partitioner.cc:1155][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.533.059 [engine_partitioner.cc:1164][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [27] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.533.080 [graph_builder.cc:865][EVENT]167074 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [239] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:33.533.366 [logger.cc:1071] 167074 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:37:33.533.396 [task_generator.cc:804][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [79] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.533.454 [task_generator.cc:805][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [45] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.534.218 [task_generator.cc:814][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [749] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.534.234 [task_generator.cc:954][EVENT]167074 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [917] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.534.293 [task_generator.cc:967][EVENT]167074 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [34] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:33.534.311 [logger.cc:1084] 167074 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:37:33.534.477 [graph_manager.cc:1152][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [1784] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.534.496 [graph_manager.cc:1164][EVENT]167074 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:33.534.530 [graph_manager.cc:1271][EVENT]167074 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [33485] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.534.541 [graph_manager.cc:1272][EVENT]167074 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:37:33.534.854 [atrace_api.c:93](tid:167074) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:37:33.534.871 [atrace_api.c:95](tid:167074) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:37:33.539.513 [graph_converter.cc:838][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1303] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.539.674 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [115] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.540.137 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [437] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.540.330 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [166] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.540.352 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [190] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.540.582 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [202] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.540.622 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.540.652 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.540.841 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [177] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.540.924 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [64] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.540.939 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [80] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.540.968 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.540.993 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.541.019 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.541.091 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [61] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.541.155 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [53] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.541.166 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [64] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.541.191 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.541.213 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.541.226 [graph_converter.cc:849][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1673] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.541.434 [graph_converter.cc:853][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [197] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.542.106 [graph_converter.cc:857][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [656] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.542.246 [graph_converter.cc:862][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [113] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.606.914 [graph_var_manager.cc:1424][EVENT]167075 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:33.607.021 [graph_manager.cc:1248][EVENT]167075 PreRun:PreRun start: graph node size 4, session id 10, graph id 9, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:33.607.280 [atrace_api.c:28](tid:167075) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:33.607.308 [trace_rb_log.c:84](tid:167075) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:33.607.320 [atrace_api.c:32](tid:167075) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:33.607.336 [client_manager.cpp:157][SetProfilingCallback][tid:167075] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:33.607.729 [parallel_partitioner.cc:165][EVENT]167075 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.607.765 [parallel_partitioner.cc:178][EVENT]167075 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.607.810 [graph_prepare.cc:1378][EVENT]167075 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.607.967 [graph_manager.cc:1050][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [173] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.607.991 [graph_manager.cc:1052][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.608.130 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.608.161 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.608.208 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.608.221 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.608.264 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.608.278 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.608.295 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.608.393 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.608.413 [graph_manager.cc:1054][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [410] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.608.621 [graph_manager.cc:1055][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [195] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.609.798 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:33.609.830 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.609.841 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.609.851 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [450] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.609.860 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.609.868 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:33.609.877 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.609.897 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.609.906 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.612.839 [graph_manager.cc:1056][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [4197] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.612.910 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.612.930 [graph_prepare.cc:1982][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [55] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.613.493 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:33.613.522 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.532 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.542 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [354] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.552 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.560 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:33.613.569 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.577 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.586 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.612 [graph_prepare.cc:1983][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [667] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.613.636 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.613.647 [graph_prepare.cc:1984][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.613.661 [graph_prepare.cc:1985][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.613.675 [graph_prepare.cc:1986][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.613.705 [graph_prepare.cc:1987][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.613.722 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.613.734 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.613.760 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.613.856 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.868 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.877 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.886 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.894 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.903 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.912 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.920 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.929 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.937 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.946 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.954 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.962 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.971 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.979 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.613.987 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.614.011 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.614.024 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.614.057 [graph_prepare.cc:1988][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [341] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.614.070 [graph_manager.cc:1065][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1195] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.627.217 [graph_manager.cc:1077][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13127] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.627.294 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.627.357 [graph_manager.cc:1080][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [98] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.638.457 [graph_manager.cc:1081][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [11083] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.638.504 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.638.520 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.638.533 [graph_manager.cc:1082][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.638.566 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.638.583 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.638.598 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.638.751 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [142] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.638.769 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.638.862 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [81] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.638.878 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.638.929 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [40] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.638.950 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.638.970 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.058 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [77] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.077 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.090 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.101 [graph_manager.cc:2700][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [540] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.343 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.639.359 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.639.369 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.639.389 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [0] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.639.399 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.639.408 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CastRemovePass is [39] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.639.417 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.639.425 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.639.434 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [10] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.639.443 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.639.451 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [17] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.639.460 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.639.468 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [21] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.639.476 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [6] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.639.485 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.639.494 [graph_manager.cc:2741][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [376] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.505 [graph_manager.cc:2752][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.529 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.542 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.564 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.579 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.592 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.605 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.626 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.641 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.655 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.671 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.684 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.697 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.719 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.732 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.742 [graph_manager.cc:2810][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [219] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.787 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.639.800 [graph_manager.cc:2821][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [48] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.639.828 [graph_manager.cc:1087][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1276] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.640.384 [graph_manager.cc:1088][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [541] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.640.448 [graph_manager.cc:1089][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.640.471 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.640.489 [graph_manager.cc:1097][EVENT]167075 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:33.640.512 [graph_manager.cc:3325][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.651.276 [engine_place.cc:144][EVENT]167075 Run:The time cost of AIcoreEngine::CheckSupported is [10544] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.651.309 [engine_place.cc:144][EVENT]167075 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.651.320 [engine_place.cc:144][EVENT]167075 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.651.413 [graph_manager.cc:3351][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10889] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.651.433 [graph_manager.cc:3364][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.651.510 [engine_partitioner.cc:1139][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [27] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.651.541 [engine_partitioner.cc:1142][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.651.723 [engine_partitioner.cc:1148][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [160] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.651.764 [engine_partitioner.cc:1155][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [27] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.651.812 [engine_partitioner.cc:1164][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.651.846 [graph_manager.cc:3405][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [400] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.651.864 [graph_manager.cc:3412][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.767.219 [graph_manager.cc:3422][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [115339] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.767.273 [graph_manager.cc:3428][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.767.453 [graph_manager.cc:3467][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [159] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.767.473 [graph_manager.cc:3377][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [116028] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.767.491 [graph_manager.cc:1106][EVENT]167075 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [126986] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.767.504 [graph_manager.cc:1115][EVENT]167075 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:33.767.530 [graph_manager.cc:1130][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.767.563 [graph_manager.cc:1131][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.767.593 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.767.614 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.767.623 [graph_manager.cc:2837][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [44] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.767.765 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [29] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.767.779 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [5] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.767.788 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondRemovePass is [6] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.767.797 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of BitcastPass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.767.806 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [8] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.767.831 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [16] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:33.767.843 [graph_manager.cc:2864][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [201] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.767.855 [graph_manager.cc:2872][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.767.876 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.767.891 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.767.909 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.767.922 [compile_nodes_pass.cc:88][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.767.932 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.767.942 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.768.051 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [100] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.768.102 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.768.117 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.768.131 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.768.145 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.768.155 [graph_manager.cc:2927][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [284] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.768.168 [graph_manager.cc:2937][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.768.183 [graph_manager.cc:2943][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.768.194 [graph_manager.cc:2950][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.768.388 [graph_manager.cc:2958][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [56] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.768.423 [graph_manager.cc:1132][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [846] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.768.503 [graph_manager.cc:1135][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [67] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.768.554 [graph_manager.cc:2975][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.768.586 [graph_manager.cc:2981][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.768.600 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.768.610 [graph_manager.cc:2986][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.768.620 [graph_manager.cc:1136][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [87] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.768.952 [graph_manager.cc:3555][EVENT]167075 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [292] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.769.084 [engine_partitioner.cc:1139][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.769.116 [engine_partitioner.cc:1142][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.769.263 [engine_partitioner.cc:1148][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [136] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.769.300 [engine_partitioner.cc:1155][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.769.345 [engine_partitioner.cc:1164][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.769.372 [graph_builder.cc:865][EVENT]167075 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [352] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:33.769.817 [logger.cc:1071] 167075 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:37:33.769.853 [task_generator.cc:804][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [86] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.769.937 [task_generator.cc:805][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [71] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.771.681 [task_generator.cc:814][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1729] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.771.696 [task_generator.cc:954][EVENT]167075 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1929] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.771.765 [task_generator.cc:967][EVENT]167075 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [38] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:33.771.791 [logger.cc:1084] 167075 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:37:33.773.250 [graph_manager.cc:1152][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4594] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.773.289 [graph_manager.cc:1164][EVENT]167075 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:33.773.343 [graph_manager.cc:1271][EVENT]167075 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [165703] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.773.355 [graph_manager.cc:1272][EVENT]167075 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:37:33.773.678 [atrace_api.c:93](tid:167075) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:37:33.773.707 [atrace_api.c:95](tid:167075) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:37:33.804.259 [graph_converter.cc:838][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [11382] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.804.481 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [162] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.806.047 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [1537] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.806.443 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [362] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.806.470 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [392] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.806.734 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [250] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.806.819 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [58] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.806.889 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [50] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.807.355 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [449] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.807.572 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [188] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.807.596 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [213] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.807.661 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [53] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.807.720 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [45] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.807.781 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [47] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.808.009 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [216] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.808.200 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [174] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.808.220 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [194] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.808.282 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [52] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.808.340 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [44] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.808.359 [graph_converter.cc:849][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4046] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.809.067 [graph_converter.cc:853][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [696] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.811.061 [graph_converter.cc:857][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1953] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.811.463 [graph_converter.cc:862][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [369] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.902.506 [graph_var_manager.cc:1424][EVENT]167076 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:33.902.604 [graph_manager.cc:1248][EVENT]167076 PreRun:PreRun start: graph node size 4, session id 11, graph id 10, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:33.903.507 [atrace_api.c:28](tid:167076) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:33.903.584 [trace_rb_log.c:84](tid:167076) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:33.903.598 [atrace_api.c:32](tid:167076) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:33.903.615 [client_manager.cpp:157][SetProfilingCallback][tid:167076] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:33.904.533 [parallel_partitioner.cc:165][EVENT]167076 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.904.574 [parallel_partitioner.cc:178][EVENT]167076 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.904.621 [graph_prepare.cc:1378][EVENT]167076 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.905.299 [graph_manager.cc:1050][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [695] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.905.328 [graph_manager.cc:1052][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.905.467 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.905.497 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.905.545 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.905.559 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.905.603 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.905.617 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.905.635 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.905.778 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.905.801 [graph_manager.cc:1054][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [460] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.906.013 [graph_manager.cc:1055][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [197] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.907.149 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:33.907.179 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.907.191 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.907.200 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of InferShapePass is [356] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.907.209 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.907.218 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:33.907.227 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [15] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.907.236 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.907.244 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of InferValuePass is [8] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.909.407 [graph_manager.cc:1056][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3353] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.909.476 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of CondRemovePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.909.495 [graph_prepare.cc:1982][EVENT]167076 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.909.998 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:33.910.026 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.037 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.047 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of InferShapePass is [308] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.056 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.065 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:33.910.073 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.082 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.090 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.115 [graph_prepare.cc:1983][EVENT]167076 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [607] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.910.150 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.910.162 [graph_prepare.cc:1984][EVENT]167076 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.910.176 [graph_prepare.cc:1985][EVENT]167076 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.910.194 [graph_prepare.cc:1986][EVENT]167076 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.910.207 [graph_prepare.cc:1987][EVENT]167076 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.910.221 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.910.233 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.910.248 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.910.341 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.353 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.363 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.372 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.380 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.389 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.398 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.407 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.415 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.423 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.432 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.440 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.448 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.456 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.465 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.483 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.910.506 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.910.518 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.910.550 [graph_prepare.cc:1988][EVENT]167076 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [333] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.910.563 [graph_manager.cc:1065][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1121] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.922.559 [graph_manager.cc:1077][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11977] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.922.631 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.922.686 [graph_manager.cc:1080][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [90] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.340 [graph_manager.cc:1081][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [4638] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.383 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.399 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.410 [graph_manager.cc:1082][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.441 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.455 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.469 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.499 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.515 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.529 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.542 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.583 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.602 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.620 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.660 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.675 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.687 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.697 [graph_manager.cc:2700][EVENT]167076 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [260] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.815 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.927.829 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of AddNPass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.927.839 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.927.848 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.927.857 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.927.866 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of CastRemovePass is [8] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.927.875 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.927.883 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.927.892 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.927.900 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.927.909 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.927.917 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.927.926 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.927.934 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.927.942 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.927.952 [graph_manager.cc:2741][EVENT]167076 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [237] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.961 [graph_manager.cc:2752][EVENT]167076 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.927.984 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.928.004 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.928.021 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.928.036 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.928.048 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.928.061 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.928.080 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.928.094 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.928.107 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.928.117 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.928.131 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.928.141 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.928.161 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.928.174 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.928.183 [graph_manager.cc:2810][EVENT]167076 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [203] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.928.213 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.928.226 [graph_manager.cc:2821][EVENT]167076 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.928.253 [graph_manager.cc:1087][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [824] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.928.386 [graph_manager.cc:1088][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [121] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.928.426 [graph_manager.cc:1089][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.928.444 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.928.460 [graph_manager.cc:1097][EVENT]167076 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:33.928.480 [graph_manager.cc:3325][EVENT]167076 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.929.529 [engine_place.cc:144][EVENT]167076 Run:The time cost of AIcoreEngine::CheckSupported is [937] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.929.559 [engine_place.cc:144][EVENT]167076 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.929.569 [engine_place.cc:144][EVENT]167076 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.929.645 [graph_manager.cc:3351][EVENT]167076 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [1143] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.929.663 [graph_manager.cc:3364][EVENT]167076 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.929.750 [engine_partitioner.cc:1139][EVENT]167076 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.929.768 [engine_partitioner.cc:1142][EVENT]167076 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.929.949 [engine_partitioner.cc:1148][EVENT]167076 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [171] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.929.994 [engine_partitioner.cc:1155][EVENT]167076 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [31] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.930.045 [engine_partitioner.cc:1164][EVENT]167076 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [39] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.930.083 [graph_manager.cc:3405][EVENT]167076 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [406] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.930.101 [graph_manager.cc:3412][EVENT]167076 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.103 [graph_manager.cc:3422][EVENT]167076 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [20988] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.153 [graph_manager.cc:3428][EVENT]167076 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.298 [graph_manager.cc:3467][EVENT]167076 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [125] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.317 [graph_manager.cc:3377][EVENT]167076 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [21642] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.335 [graph_manager.cc:1106][EVENT]167076 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [22861] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.348 [graph_manager.cc:1115][EVENT]167076 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:33.951.371 [graph_manager.cc:1130][EVENT]167076 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.403 [graph_manager.cc:1131][EVENT]167076 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.430 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.459 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.469 [graph_manager.cc:2837][EVENT]167076 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [48] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.556 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.951.570 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.951.580 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.951.589 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.951.598 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.951.607 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:33.951.616 [graph_manager.cc:2864][EVENT]167076 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [131] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.628 [graph_manager.cc:2872][EVENT]167076 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.648 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.662 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.678 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.691 [compile_nodes_pass.cc:88][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.701 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.711 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.800 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [79] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.827 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.841 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.854 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.867 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.884 [graph_manager.cc:2927][EVENT]167076 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [239] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.897 [graph_manager.cc:2937][EVENT]167076 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.910 [graph_manager.cc:2943][EVENT]167076 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.951.922 [graph_manager.cc:2950][EVENT]167076 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.952.097 [graph_manager.cc:2958][EVENT]167076 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [39] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.952.129 [graph_manager.cc:1132][EVENT]167076 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [711] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.952.201 [graph_manager.cc:1135][EVENT]167076 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [59] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.952.239 [graph_manager.cc:2975][EVENT]167076 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.952.270 [graph_manager.cc:2981][EVENT]167076 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.952.285 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.952.295 [graph_manager.cc:2986][EVENT]167076 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.952.305 [graph_manager.cc:1136][EVENT]167076 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [87] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.952.435 [graph_manager.cc:3555][EVENT]167076 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [95] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.952.528 [engine_partitioner.cc:1139][EVENT]167076 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.952.545 [engine_partitioner.cc:1142][EVENT]167076 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.952.700 [engine_partitioner.cc:1148][EVENT]167076 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [144] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.952.736 [engine_partitioner.cc:1155][EVENT]167076 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.952.779 [engine_partitioner.cc:1164][EVENT]167076 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.952.802 [graph_builder.cc:865][EVENT]167076 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [312] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:33.953.229 [logger.cc:1071] 167076 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:37:33.953.264 [task_generator.cc:804][EVENT]167076 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [170] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.953.343 [task_generator.cc:805][EVENT]167076 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [56] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.954.110 [task_generator.cc:814][EVENT]167076 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [752] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.954.127 [task_generator.cc:954][EVENT]167076 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1033] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.954.184 [task_generator.cc:967][EVENT]167076 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [30] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:33.954.204 [logger.cc:1084] 167076 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:37:33.957.182 [graph_manager.cc:1152][EVENT]167076 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4850] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.957.224 [graph_manager.cc:1164][EVENT]167076 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:33.957.260 [graph_manager.cc:1271][EVENT]167076 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [52817] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.957.271 [graph_manager.cc:1272][EVENT]167076 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:37:33.957.590 [atrace_api.c:93](tid:167076) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:37:33.957.609 [atrace_api.c:95](tid:167076) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:37:33.978.232 [graph_converter.cc:838][EVENT]167076 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [6690] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.978.455 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of ZeroCopy is [168] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.979.041 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of CEM is [557] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.979.276 [copy_flow_launch_fuse.cc:395][EVENT]167076 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [204] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.979.300 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [230] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.979.608 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [294] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.979.654 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [24] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.979.690 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of ZeroCopy is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.979.908 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of CEM is [205] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.980.001 [copy_flow_launch_fuse.cc:395][EVENT]167076 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [71] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.980.015 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [86] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.980.046 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.980.074 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.980.106 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of ZeroCopy is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.980.210 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of CEM is [73] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.980.286 [copy_flow_launch_fuse.cc:395][EVENT]167076 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [61] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.980.301 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [75] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.980.329 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.980.354 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.980.367 [graph_converter.cc:849][EVENT]167076 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [2085] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.980.610 [graph_converter.cc:853][EVENT]167076 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [233] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.981.397 [graph_converter.cc:857][EVENT]167076 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [769] micro second. [INFO] GE(164043,python):2024-01-10-11:37:33.981.559 [graph_converter.cc:862][EVENT]167076 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [131] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.047.501 [graph_var_manager.cc:1424][EVENT]167074 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:34.047.602 [graph_manager.cc:1248][EVENT]167074 PreRun:PreRun start: graph node size 4, session id 12, graph id 11, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:34.047.866 [atrace_api.c:28](tid:167074) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:34.047.893 [trace_rb_log.c:84](tid:167074) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:34.047.906 [atrace_api.c:32](tid:167074) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:34.047.923 [client_manager.cpp:157][SetProfilingCallback][tid:167074] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:34.048.295 [parallel_partitioner.cc:165][EVENT]167074 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.048.333 [parallel_partitioner.cc:178][EVENT]167074 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.048.379 [graph_prepare.cc:1378][EVENT]167074 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.048.544 [graph_manager.cc:1050][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [182] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.048.568 [graph_manager.cc:1052][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.048.701 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.048.731 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.048.807 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.048.820 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.048.878 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.048.891 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.048.908 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.049.011 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.049.031 [graph_manager.cc:1054][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [452] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.049.247 [graph_manager.cc:1055][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [203] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.050.313 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:34.050.343 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.050.355 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.050.365 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [328] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.050.374 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.050.383 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:34.050.392 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [22] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.050.401 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.050.409 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [9] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.052.529 [graph_manager.cc:1056][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3262] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.052.597 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.052.616 [graph_prepare.cc:1982][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [52] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.053.071 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:34.053.098 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.109 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.130 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [262] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.140 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.148 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:34.053.157 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.165 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.174 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.199 [graph_prepare.cc:1983][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [570] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.053.223 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.053.234 [graph_prepare.cc:1984][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.053.248 [graph_prepare.cc:1985][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.053.262 [graph_prepare.cc:1986][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.053.274 [graph_prepare.cc:1987][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.053.289 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.053.301 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.053.316 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.053.407 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.418 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.427 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.436 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.444 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.453 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.461 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.469 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.486 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.495 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.503 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.512 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.520 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.528 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.537 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.545 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.053.567 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.053.581 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.053.614 [graph_prepare.cc:1988][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [331] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.053.627 [graph_manager.cc:1065][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1065] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.067.120 [graph_manager.cc:1077][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13474] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.067.195 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.067.247 [graph_manager.cc:1080][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [86] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.075.893 [graph_manager.cc:1081][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [8629] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.075.939 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.075.956 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.075.968 [graph_manager.cc:1082][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.075.999 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.013 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.039 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.142 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [92] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.158 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.206 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.222 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.261 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [29] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.280 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.299 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.326 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.341 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.354 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.363 [graph_manager.cc:2700][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [369] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.489 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.076.503 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.076.513 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.076.522 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.076.530 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.076.539 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CastRemovePass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.076.547 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.076.556 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.076.564 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.076.572 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.076.581 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.076.598 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.076.606 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.076.615 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.076.623 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.076.633 [graph_manager.cc:2741][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [251] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.642 [graph_manager.cc:2752][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.664 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.676 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.693 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.709 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.720 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.733 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.752 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.766 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.780 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.791 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.803 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.814 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.831 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.844 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.853 [graph_manager.cc:2810][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [194] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.883 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.076.894 [graph_manager.cc:2821][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.076.927 [graph_manager.cc:1087][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [941] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.077.064 [graph_manager.cc:1088][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [123] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.077.105 [graph_manager.cc:1089][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.077.124 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.077.139 [graph_manager.cc:1097][EVENT]167074 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:34.077.160 [graph_manager.cc:3325][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.077.599 [engine_place.cc:144][EVENT]167074 Run:The time cost of AIcoreEngine::CheckSupported is [294] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.077.627 [engine_place.cc:144][EVENT]167074 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.077.637 [engine_place.cc:144][EVENT]167074 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.077.724 [graph_manager.cc:3351][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [550] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.077.744 [graph_manager.cc:3364][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.077.810 [engine_partitioner.cc:1139][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.077.828 [engine_partitioner.cc:1142][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.077.983 [engine_partitioner.cc:1148][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [145] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.078.027 [engine_partitioner.cc:1155][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [31] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.078.075 [engine_partitioner.cc:1164][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.078.107 [graph_manager.cc:3405][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [350] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.078.126 [graph_manager.cc:3412][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.096.299 [graph_manager.cc:3422][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [18158] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.096.352 [graph_manager.cc:3428][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.096.517 [graph_manager.cc:3467][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [126] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.096.540 [graph_manager.cc:3377][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [18785] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.096.560 [graph_manager.cc:1106][EVENT]167074 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [19407] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.096.577 [graph_manager.cc:1115][EVENT]167074 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:34.096.605 [graph_manager.cc:1130][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.096.642 [graph_manager.cc:1131][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.096.671 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.096.690 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.096.700 [graph_manager.cc:2837][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [39] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.096.789 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.096.806 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.096.816 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.096.826 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.096.835 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.096.845 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [7] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.096.858 [graph_manager.cc:2864][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [138] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.096.870 [graph_manager.cc:2872][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.096.893 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.096.909 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.096.927 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.096.941 [compile_nodes_pass.cc:88][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.096.960 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.096.970 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.097.053 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [73] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.097.083 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.097.097 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.097.110 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.097.123 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.097.131 [graph_manager.cc:2927][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [242] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.097.145 [graph_manager.cc:2937][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.097.159 [graph_manager.cc:2943][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.097.170 [graph_manager.cc:2950][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.097.397 [graph_manager.cc:2958][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [41] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.097.433 [graph_manager.cc:1132][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [774] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.097.511 [graph_manager.cc:1135][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [61] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.097.544 [graph_manager.cc:2975][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.097.575 [graph_manager.cc:2981][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.097.589 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.097.599 [graph_manager.cc:2986][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.097.608 [graph_manager.cc:1136][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [80] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.097.761 [graph_manager.cc:3555][EVENT]167074 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [119] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.097.860 [engine_partitioner.cc:1139][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.097.890 [engine_partitioner.cc:1142][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.098.014 [engine_partitioner.cc:1148][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [114] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.098.050 [engine_partitioner.cc:1155][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.098.092 [engine_partitioner.cc:1164][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.098.118 [graph_builder.cc:865][EVENT]167074 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [293] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:34.098.427 [logger.cc:1071] 167074 ModelBindStream: model_id=1856, stream_id=65, flag=0. [INFO] GE(164043,python):2024-01-10-11:37:34.098.461 [task_generator.cc:804][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [83] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.098.524 [task_generator.cc:805][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [50] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.099.217 [task_generator.cc:814][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [677] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.099.232 [task_generator.cc:954][EVENT]167074 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [855] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.099.291 [task_generator.cc:967][EVENT]167074 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [32] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:34.099.308 [logger.cc:1084] 167074 ModelUnbindStream: model_id=1856, stream_id=65, [INFO] GE(164043,python):2024-01-10-11:37:34.099.962 [graph_manager.cc:1152][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2328] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.099.997 [graph_manager.cc:1164][EVENT]167074 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:34.100.034 [graph_manager.cc:1271][EVENT]167074 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [51827] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.100.044 [graph_manager.cc:1272][EVENT]167074 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:37:34.100.356 [atrace_api.c:93](tid:167074) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:37:34.100.373 [atrace_api.c:95](tid:167074) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:37:34.112.008 [graph_converter.cc:838][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [3800] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.112.176 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [118] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.112.671 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [472] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.112.886 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [187] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.112.911 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [213] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.113.134 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [208] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.113.189 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.113.224 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.113.428 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [190] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.113.517 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [67] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.113.534 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [84] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.113.564 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.113.589 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.113.616 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.113.756 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [129] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.113.829 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [59] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.113.842 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [72] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.113.869 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.113.896 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.113.914 [graph_converter.cc:849][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1862] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.114.141 [graph_converter.cc:853][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [217] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.114.871 [graph_converter.cc:857][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [711] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.115.023 [graph_converter.cc:862][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [124] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.191.431 [graph_var_manager.cc:1424][EVENT]167075 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:34.191.530 [graph_manager.cc:1248][EVENT]167075 PreRun:PreRun start: graph node size 4, session id 13, graph id 12, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:34.191.770 [atrace_api.c:28](tid:167075) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:34.191.801 [trace_rb_log.c:84](tid:167075) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:34.191.814 [atrace_api.c:32](tid:167075) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:34.191.832 [client_manager.cpp:157][SetProfilingCallback][tid:167075] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:34.192.186 [parallel_partitioner.cc:165][EVENT]167075 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.192.250 [parallel_partitioner.cc:178][EVENT]167075 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.192.296 [graph_prepare.cc:1378][EVENT]167075 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.192.434 [graph_manager.cc:1050][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [154] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.192.457 [graph_manager.cc:1052][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.192.607 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.192.638 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.192.685 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.192.698 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.192.740 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.192.753 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.192.770 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.192.869 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.192.889 [graph_manager.cc:1054][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [419] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.193.099 [graph_manager.cc:1055][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [196] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.194.266 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:34.194.297 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.194.309 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.194.319 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [361] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.194.328 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.194.337 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:34.194.345 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.194.354 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.194.373 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.196.523 [graph_manager.cc:1056][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3405] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.196.592 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.196.611 [graph_prepare.cc:1982][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.197.055 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:34.197.082 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.094 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.103 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [247] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.112 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.121 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:34.197.129 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.138 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.146 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.172 [graph_prepare.cc:1983][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [548] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.197.194 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.197.206 [graph_prepare.cc:1984][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.197.219 [graph_prepare.cc:1985][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.197.234 [graph_prepare.cc:1986][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.197.245 [graph_prepare.cc:1987][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.197.260 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.197.272 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.197.287 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.197.389 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.401 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.410 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.418 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.427 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.436 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.444 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.453 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.461 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.470 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.479 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.487 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SnapshotPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.495 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.504 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [7] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.512 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.520 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.197.543 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.197.556 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.197.589 [graph_prepare.cc:1988][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [335] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.197.603 [graph_manager.cc:1065][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1045] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.210.566 [graph_manager.cc:1077][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12942] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.210.629 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.210.679 [graph_manager.cc:1080][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [76] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.170 [graph_manager.cc:1081][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3464] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.214 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.229 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.240 [graph_manager.cc:1082][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.272 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.287 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.301 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.410 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [99] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.432 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.485 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [43] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.500 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.540 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.559 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.585 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.613 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.629 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.642 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.652 [graph_manager.cc:2700][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [385] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.771 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.214.785 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.214.794 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.214.814 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.214.823 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.214.832 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.214.841 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.214.849 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.214.857 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.214.866 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.214.875 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.214.883 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.214.891 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.214.899 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.214.908 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.214.917 [graph_manager.cc:2741][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [247] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.926 [graph_manager.cc:2752][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.948 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.960 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.976 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.214.990 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.001 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.014 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.033 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.046 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.059 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.075 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.089 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.101 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.120 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.134 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.143 [graph_manager.cc:2810][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [198] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.173 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.215.185 [graph_manager.cc:2821][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.212 [graph_manager.cc:1087][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [953] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.345 [graph_manager.cc:1088][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [120] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.386 [graph_manager.cc:1089][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.403 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.419 [graph_manager.cc:1097][EVENT]167075 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:34.215.439 [graph_manager.cc:3325][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.826 [engine_place.cc:144][EVENT]167075 Run:The time cost of AIcoreEngine::CheckSupported is [284] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.855 [engine_place.cc:144][EVENT]167075 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.864 [engine_place.cc:144][EVENT]167075 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.949 [graph_manager.cc:3351][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [497] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.215.968 [graph_manager.cc:3364][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.216.038 [engine_partitioner.cc:1139][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.216.056 [engine_partitioner.cc:1142][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.216.207 [engine_partitioner.cc:1148][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [141] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.216.262 [engine_partitioner.cc:1155][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.216.309 [engine_partitioner.cc:1164][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.216.341 [graph_manager.cc:3405][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [361] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.216.359 [graph_manager.cc:3412][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.316.914 [graph_manager.cc:3422][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [100540] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.316.990 [graph_manager.cc:3428][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.181 [graph_manager.cc:3467][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [165] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.204 [graph_manager.cc:3377][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [101224] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.224 [graph_manager.cc:1106][EVENT]167075 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [101790] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.238 [graph_manager.cc:1115][EVENT]167075 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:34.317.263 [graph_manager.cc:1130][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.300 [graph_manager.cc:1131][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.328 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.345 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.355 [graph_manager.cc:2837][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.456 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [21] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.317.470 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.317.480 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.317.489 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.317.498 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.317.507 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.317.536 [graph_manager.cc:2864][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [161] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.549 [graph_manager.cc:2872][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.571 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.585 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.601 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.614 [compile_nodes_pass.cc:88][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.624 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.634 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.743 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [100] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.789 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [31] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.804 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.818 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.833 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.842 [graph_manager.cc:2927][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [276] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.855 [graph_manager.cc:2937][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.871 [graph_manager.cc:2943][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.317.882 [graph_manager.cc:2950][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.318.075 [graph_manager.cc:2958][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [45] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.318.110 [graph_manager.cc:1132][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [795] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.318.195 [graph_manager.cc:1135][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [72] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.318.242 [graph_manager.cc:2975][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.318.276 [graph_manager.cc:2981][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.318.291 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.318.301 [graph_manager.cc:2986][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.318.310 [graph_manager.cc:1136][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [87] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.318.456 [graph_manager.cc:3555][EVENT]167075 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [110] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.318.553 [engine_partitioner.cc:1139][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.318.571 [engine_partitioner.cc:1142][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.318.709 [engine_partitioner.cc:1148][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [128] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.318.746 [engine_partitioner.cc:1155][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [24] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.318.792 [engine_partitioner.cc:1164][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.318.817 [graph_builder.cc:865][EVENT]167075 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [303] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:34.319.166 [logger.cc:1071] 167075 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:37:34.319.199 [task_generator.cc:804][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [84] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.319.265 [task_generator.cc:805][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [53] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.320.188 [task_generator.cc:814][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [908] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.320.204 [task_generator.cc:954][EVENT]167075 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1090] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.320.271 [task_generator.cc:967][EVENT]167075 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [36] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:34.320.291 [logger.cc:1084] 167075 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:37:34.322.062 [graph_manager.cc:1152][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3723] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.322.107 [graph_manager.cc:1164][EVENT]167075 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:34.322.148 [graph_manager.cc:1271][EVENT]167075 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [130051] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.322.173 [graph_manager.cc:1272][EVENT]167075 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:37:34.322.504 [atrace_api.c:93](tid:167075) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:37:34.322.523 [atrace_api.c:95](tid:167075) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:37:34.354.073 [graph_converter.cc:838][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [10252] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.354.262 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [133] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.355.006 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [719] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.355.253 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [219] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.355.278 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [245] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.355.477 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [185] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.355.528 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [31] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.355.569 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [25] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.355.837 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [255] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.355.952 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [94] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.355.969 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [111] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.356.005 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [27] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.356.038 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.356.073 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [24] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.356.188 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [104] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.356.283 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [82] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.356.295 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [95] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.356.329 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [24] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.356.360 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.356.375 [graph_converter.cc:849][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [2251] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.356.716 [graph_converter.cc:853][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [331] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.357.726 [graph_converter.cc:857][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [990] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.357.966 [graph_converter.cc:862][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [184] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.432.236 [graph_var_manager.cc:1424][EVENT]167073 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:34.432.338 [graph_manager.cc:1248][EVENT]167073 PreRun:PreRun start: graph node size 4, session id 14, graph id 13, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:34.432.598 [atrace_api.c:28](tid:167073) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:34.432.625 [trace_rb_log.c:84](tid:167073) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:34.432.639 [atrace_api.c:32](tid:167073) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:34.432.655 [client_manager.cpp:157][SetProfilingCallback][tid:167073] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:34.433.011 [parallel_partitioner.cc:165][EVENT]167073 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.433.049 [parallel_partitioner.cc:178][EVENT]167073 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.433.095 [graph_prepare.cc:1378][EVENT]167073 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.433.244 [graph_manager.cc:1050][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [166] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.433.267 [graph_manager.cc:1052][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.433.397 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.433.427 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.433.475 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.433.488 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.433.532 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.433.546 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.433.562 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.433.662 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.433.681 [graph_manager.cc:1054][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [403] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.433.905 [graph_manager.cc:1055][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [196] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.435.049 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:34.435.080 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.435.091 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.435.100 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferShapePass is [435] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.435.109 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.435.118 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:34.435.126 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [9] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.435.135 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.435.144 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.438.084 [graph_manager.cc:1056][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [4133] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.438.156 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.438.176 [graph_prepare.cc:1982][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [55] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.438.718 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:34.438.748 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.438.759 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.438.768 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferShapePass is [342] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.438.777 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.438.786 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:34.438.794 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.438.803 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.438.811 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.438.838 [graph_prepare.cc:1983][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [647] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.438.861 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.438.884 [graph_prepare.cc:1984][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [31] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.438.899 [graph_prepare.cc:1985][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.438.915 [graph_prepare.cc:1986][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.438.926 [graph_prepare.cc:1987][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.438.940 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.438.952 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.438.967 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.439.059 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.439.071 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.439.080 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.439.088 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.439.097 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.439.105 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.439.114 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.439.122 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.439.131 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.439.140 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.439.148 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.439.156 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.439.165 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.439.173 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.439.181 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.439.190 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.439.229 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.439.242 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.439.275 [graph_prepare.cc:1988][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [340] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.439.288 [graph_manager.cc:1065][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1167] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.452.393 [graph_manager.cc:1077][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13084] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.452.464 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.452.512 [graph_manager.cc:1080][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [82] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.463.651 [graph_manager.cc:1081][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [11124] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.463.696 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.463.711 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.463.723 [graph_manager.cc:1082][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.463.755 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.463.770 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.463.784 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.463.940 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [145] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.463.958 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.052 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [83] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.070 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.119 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.141 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.161 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.261 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [77] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.278 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.291 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.301 [graph_manager.cc:2700][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [553] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.540 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of EnterPass is [0] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.464.556 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AddNPass is [4] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.464.567 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.464.576 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [5] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.464.586 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.464.595 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CastRemovePass is [40] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.464.603 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.464.612 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.464.621 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [10] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.464.629 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.464.637 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [18] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.464.646 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.464.654 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.464.662 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [9] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.464.671 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [5] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.464.681 [graph_manager.cc:2741][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [361] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.690 [graph_manager.cc:2752][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.714 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.727 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.757 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.773 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.786 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.799 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.818 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.833 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.848 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.858 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.872 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.884 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.906 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.919 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.930 [graph_manager.cc:2810][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [220] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.464.975 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.464.987 [graph_manager.cc:2821][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [48] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.465.015 [graph_manager.cc:1087][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1272] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.465.573 [graph_manager.cc:1088][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [544] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.465.636 [graph_manager.cc:1089][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [31] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.465.660 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.465.677 [graph_manager.cc:1097][EVENT]167073 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:34.465.715 [graph_manager.cc:3325][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.476.557 [engine_place.cc:144][EVENT]167073 Run:The time cost of AIcoreEngine::CheckSupported is [10590] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.476.605 [engine_place.cc:144][EVENT]167073 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.476.619 [engine_place.cc:144][EVENT]167073 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.476.714 [graph_manager.cc:3351][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10981] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.476.736 [graph_manager.cc:3364][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.476.815 [engine_partitioner.cc:1139][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [27] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.476.849 [engine_partitioner.cc:1142][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.477.019 [engine_partitioner.cc:1148][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [160] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.477.063 [engine_partitioner.cc:1155][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [28] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.477.114 [engine_partitioner.cc:1164][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.477.153 [graph_manager.cc:3405][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [400] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.477.172 [graph_manager.cc:3412][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.591.659 [graph_manager.cc:3422][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [114472] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.591.712 [graph_manager.cc:3428][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.591.895 [graph_manager.cc:3467][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [160] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.591.916 [graph_manager.cc:3377][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [115164] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.591.934 [graph_manager.cc:1106][EVENT]167073 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [126225] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.591.946 [graph_manager.cc:1115][EVENT]167073 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:34.591.971 [graph_manager.cc:1130][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.005 [graph_manager.cc:1131][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.034 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.066 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.076 [graph_manager.cc:2837][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [55] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.215 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [26] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.592.229 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.592.239 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.592.248 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of BitcastPass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.592.257 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.592.266 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [18] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.592.275 [graph_manager.cc:2864][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [180] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.287 [graph_manager.cc:2872][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.308 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.323 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.340 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.355 [compile_nodes_pass.cc:88][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.365 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.375 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.485 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [101] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.538 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [40] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.552 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.567 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.582 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.600 [graph_manager.cc:2927][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [296] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.613 [graph_manager.cc:2937][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.628 [graph_manager.cc:2943][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.639 [graph_manager.cc:2950][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.834 [graph_manager.cc:2958][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [58] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.868 [graph_manager.cc:1132][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [849] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.949 [graph_manager.cc:1135][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [68] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.592.986 [graph_manager.cc:2975][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.593.018 [graph_manager.cc:2981][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.593.032 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.593.043 [graph_manager.cc:2986][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.593.053 [graph_manager.cc:1136][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [87] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.593.383 [graph_manager.cc:3555][EVENT]167073 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [288] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.593.517 [engine_partitioner.cc:1139][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.593.545 [engine_partitioner.cc:1142][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.593.747 [engine_partitioner.cc:1148][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [191] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.593.786 [engine_partitioner.cc:1155][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.593.830 [engine_partitioner.cc:1164][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.593.856 [graph_builder.cc:865][EVENT]167073 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [403] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:34.594.248 [logger.cc:1071] 167073 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:37:34.594.283 [task_generator.cc:804][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [80] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.594.380 [task_generator.cc:805][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [74] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.596.046 [task_generator.cc:814][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1650] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.596.062 [task_generator.cc:954][EVENT]167073 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1859] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.596.132 [task_generator.cc:967][EVENT]167073 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [38] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:34.596.158 [logger.cc:1084] 167073 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:37:34.597.488 [graph_manager.cc:1152][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4400] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.597.528 [graph_manager.cc:1164][EVENT]167073 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:34.597.567 [graph_manager.cc:1271][EVENT]167073 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [164642] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.597.578 [graph_manager.cc:1272][EVENT]167073 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:37:34.597.954 [atrace_api.c:93](tid:167073) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:37:34.597.976 [atrace_api.c:95](tid:167073) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:37:34.628.307 [graph_converter.cc:838][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [11336] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.628.530 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [164] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.630.113 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [1554] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.630.519 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [370] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.630.549 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [403] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.630.814 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [252] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.630.901 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [61] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.630.971 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [50] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.631.439 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [451] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.631.661 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [191] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.631.686 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [217] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.631.752 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [55] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.631.813 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [45] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.631.873 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [46] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.632.115 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [216] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.632.314 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [179] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.632.333 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [200] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.632.397 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [53] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.632.456 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [46] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.632.475 [graph_converter.cc:849][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4116] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.633.187 [graph_converter.cc:853][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [701] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.635.184 [graph_converter.cc:857][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1970] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.635.589 [graph_converter.cc:862][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [370] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.720.714 [graph_var_manager.cc:1424][EVENT]167075 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:34.720.814 [graph_manager.cc:1248][EVENT]167075 PreRun:PreRun start: graph node size 4, session id 15, graph id 14, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:34.721.052 [atrace_api.c:28](tid:167075) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:34.721.082 [trace_rb_log.c:84](tid:167075) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:34.721.095 [atrace_api.c:32](tid:167075) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:34.721.111 [client_manager.cpp:157][SetProfilingCallback][tid:167075] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:34.721.446 [parallel_partitioner.cc:165][EVENT]167075 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.721.482 [parallel_partitioner.cc:178][EVENT]167075 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.721.528 [graph_prepare.cc:1378][EVENT]167075 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.721.661 [graph_manager.cc:1050][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [150] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.721.684 [graph_manager.cc:1052][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.721.863 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.721.894 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.721.941 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.721.986 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.722.029 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.722.043 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.722.062 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.722.162 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.722.183 [graph_manager.cc:1054][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [443] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.722.396 [graph_manager.cc:1055][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [200] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.723.515 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:34.723.545 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.723.556 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.723.566 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [432] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.723.575 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.723.584 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:34.723.593 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.723.601 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.723.610 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.726.434 [graph_manager.cc:1056][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [4018] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.726.504 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.726.523 [graph_prepare.cc:1982][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [54] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.727.062 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:34.727.090 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.101 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.122 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [336] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.131 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.140 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:34.727.148 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.157 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.165 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.191 [graph_prepare.cc:1983][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [654] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.727.216 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.727.227 [graph_prepare.cc:1984][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.727.241 [graph_prepare.cc:1985][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.727.256 [graph_prepare.cc:1986][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.727.268 [graph_prepare.cc:1987][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.727.283 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.727.295 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.727.309 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.727.400 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.413 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.423 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.431 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.440 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DropOutPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.448 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.457 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.465 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.482 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.491 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.500 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.508 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.517 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.525 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.533 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.541 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:34.727.564 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.727.578 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.727.611 [graph_prepare.cc:1988][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [334] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.727.625 [graph_manager.cc:1065][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1155] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.740.678 [graph_manager.cc:1077][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13033] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.740.750 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.740.798 [graph_manager.cc:1080][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [82] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.751.791 [graph_manager.cc:1081][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [10977] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.751.838 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.751.854 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.751.866 [graph_manager.cc:1082][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.751.900 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.751.916 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.751.931 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.097 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [141] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.115 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.206 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [79] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.221 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.271 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [40] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.292 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.313 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.401 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [77] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.418 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.431 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.442 [graph_manager.cc:2700][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [547] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.679 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.752.694 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.752.704 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.752.714 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.752.723 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.752.731 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CastRemovePass is [39] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.752.740 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.752.748 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.752.757 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [10] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.752.765 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.752.774 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [18] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.752.792 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.752.801 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.752.810 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [11] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.752.818 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.752.828 [graph_manager.cc:2741][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [368] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.837 [graph_manager.cc:2752][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.861 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.874 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.896 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.913 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.924 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.938 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.958 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.972 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.986 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.752.995 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.753.008 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.753.020 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.753.042 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.753.056 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.753.064 [graph_manager.cc:2810][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [208] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.753.108 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.753.119 [graph_manager.cc:2821][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [46] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.753.155 [graph_manager.cc:1087][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1268] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.753.726 [graph_manager.cc:1088][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [558] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.753.789 [graph_manager.cc:1089][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [31] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.753.811 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.753.830 [graph_manager.cc:1097][EVENT]167075 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:34.753.854 [graph_manager.cc:3325][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.764.363 [engine_place.cc:144][EVENT]167075 Run:The time cost of AIcoreEngine::CheckSupported is [10287] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.764.398 [engine_place.cc:144][EVENT]167075 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.764.413 [engine_place.cc:144][EVENT]167075 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.764.508 [graph_manager.cc:3351][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10641] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.764.530 [graph_manager.cc:3364][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.764.611 [engine_partitioner.cc:1139][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [28] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.764.644 [engine_partitioner.cc:1142][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.764.815 [engine_partitioner.cc:1148][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [156] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.764.860 [engine_partitioner.cc:1155][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [27] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.764.907 [engine_partitioner.cc:1164][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.764.946 [graph_manager.cc:3405][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [399] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.764.966 [graph_manager.cc:3412][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.008 [graph_manager.cc:3422][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [116023] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.059 [graph_manager.cc:3428][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.260 [graph_manager.cc:3467][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [154] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.280 [graph_manager.cc:3377][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [116734] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.297 [graph_manager.cc:1106][EVENT]167075 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [127451] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.311 [graph_manager.cc:1115][EVENT]167075 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:34.881.336 [graph_manager.cc:1130][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.371 [graph_manager.cc:1131][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.401 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.419 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.429 [graph_manager.cc:2837][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [41] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.568 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [25] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.881.582 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.881.592 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.881.601 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.881.610 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [7] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.881.618 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [14] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:34.881.628 [graph_manager.cc:2864][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [180] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.641 [graph_manager.cc:2872][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.660 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.675 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.704 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.721 [compile_nodes_pass.cc:88][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.733 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.753 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.865 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [101] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.917 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [40] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.932 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.946 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.961 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.970 [graph_manager.cc:2927][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [315] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.983 [graph_manager.cc:2937][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.881.998 [graph_manager.cc:2943][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.882.010 [graph_manager.cc:2950][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.882.206 [graph_manager.cc:2958][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [56] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.882.241 [graph_manager.cc:1132][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [857] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.882.320 [graph_manager.cc:1135][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [66] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.882.359 [graph_manager.cc:2975][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.882.389 [graph_manager.cc:2981][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.882.405 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.882.416 [graph_manager.cc:2986][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.882.425 [graph_manager.cc:1136][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [87] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.882.759 [graph_manager.cc:3555][EVENT]167075 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [292] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.882.893 [engine_partitioner.cc:1139][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.882.934 [engine_partitioner.cc:1142][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.883.078 [engine_partitioner.cc:1148][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [133] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.883.118 [engine_partitioner.cc:1155][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.883.162 [engine_partitioner.cc:1164][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.883.190 [graph_builder.cc:865][EVENT]167075 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [360] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:34.883.584 [logger.cc:1071] 167075 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:37:34.883.621 [task_generator.cc:804][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [91] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.883.705 [task_generator.cc:805][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [70] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.885.430 [task_generator.cc:814][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1707] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.885.449 [task_generator.cc:954][EVENT]167075 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1921] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.885.521 [task_generator.cc:967][EVENT]167075 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [38] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:34.885.551 [logger.cc:1084] 167075 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:37:34.886.892 [graph_manager.cc:1152][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4433] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.886.933 [graph_manager.cc:1164][EVENT]167075 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:34.886.971 [graph_manager.cc:1271][EVENT]167075 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [165610] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.886.983 [graph_manager.cc:1272][EVENT]167075 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:37:34.887.308 [atrace_api.c:93](tid:167075) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:37:34.887.324 [atrace_api.c:95](tid:167075) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:37:34.916.389 [graph_converter.cc:838][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [11065] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.916.616 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [169] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.918.204 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [1560] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.918.603 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [365] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.918.631 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [397] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.918.893 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [248] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.918.997 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [62] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.919.067 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [51] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.919.527 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [444] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.919.745 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [190] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.919.769 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [215] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.919.835 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [54] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.919.894 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [45] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.919.953 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [47] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.920.174 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [209] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.920.367 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [176] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.920.386 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [196] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.920.448 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [51] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.920.504 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [44] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.920.523 [graph_converter.cc:849][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4081] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.921.236 [graph_converter.cc:853][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [701] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.923.158 [graph_converter.cc:857][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1898] micro second. [INFO] GE(164043,python):2024-01-10-11:37:34.923.561 [graph_converter.cc:862][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [371] micro second. [INFO] HCCP(164043,python):2024-01-10-11:37:34.940.965 [ra_host.c:1761]tid:168500,ra_socket_white_list_add(1761) : Input parameters: phy_id[2], local_ip[2.0.0.0], num[1] [INFO] HCCP(164043,python):2024-01-10-11:37:34.941.250 [ra_host.c:1761]tid:168500,ra_socket_white_list_add(1761) : Input parameters: phy_id[2], local_ip[2.0.0.0], num[1] [INFO] HCCP(164043,python):2024-01-10-11:37:34.941.343 [ra_host.c:825]tid:168500,ra_socket_batch_connect(825) : Input parameters: [0]th, phy_id[2], local_ip[2.0.0.0], remote_ip[1.0.0.0], tag:[8.92.9.85%enp189s0f0_60000_0_1704857845691656] [INFO] HCCP(164043,python):2024-01-10-11:37:35.509.647 [ra_host.c:1761]tid:167075,ra_socket_white_list_add(1761) : Input parameters: phy_id[2], local_ip[2.0.0.0], num[1] [INFO] HCCP(164043,python):2024-01-10-11:37:35.509.851 [ra_host.c:825]tid:167075,ra_socket_batch_connect(825) : Input parameters: [0]th, phy_id[2], local_ip[2.0.0.0], remote_ip[3.0.0.0], tag:[HeartBeat_8.92.9.85/2_to_8.92.9.85/3] [INFO] HCCL(164043,python):2024-01-10-11:37:35.515.953 [hccl_impl.cc:3258][167075]resource creation success, take time [576279]us, tag[AllReduce_8.92.9.85%enp189s0f0_60000_0_1704857845691656] [INFO] GE(164043,python):2024-01-10-11:37:35.599.204 [graph_var_manager.cc:1424][EVENT]167075 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:35.599.300 [graph_manager.cc:1248][EVENT]167075 PreRun:PreRun start: graph node size 3, session id 16, graph id 15, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:35.599.566 [atrace_api.c:28](tid:167075) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:35.599.591 [trace_rb_log.c:84](tid:167075) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:35.599.604 [atrace_api.c:32](tid:167075) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:35.599.621 [client_manager.cpp:157][SetProfilingCallback][tid:167075] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:35.599.982 [parallel_partitioner.cc:165][EVENT]167075 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.600.018 [parallel_partitioner.cc:178][EVENT]167075 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.600.062 [graph_prepare.cc:1378][EVENT]167075 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.600.190 [graph_manager.cc:1050][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [142] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.600.212 [graph_manager.cc:1052][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.600.331 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.600.360 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.600.407 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.600.419 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.600.461 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.600.474 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.600.491 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.600.587 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.600.610 [graph_manager.cc:1054][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [385] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.600.824 [graph_manager.cc:1055][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [196] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.601.726 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:35.601.780 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.601.792 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.601.802 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [277] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.601.812 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.601.821 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:35.601.829 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [10] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.601.838 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.601.847 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.603.807 [graph_manager.cc:1056][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2958] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.603.874 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.603.894 [graph_prepare.cc:1982][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [52] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.604.267 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:35.604.293 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.308 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.318 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [170] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.331 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.340 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [6] [INFO] GE(164043,python):2024-01-10-11:37:35.604.352 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.364 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.373 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.422 [graph_prepare.cc:1983][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [513] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.604.450 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.604.463 [graph_prepare.cc:1984][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.604.487 [graph_prepare.cc:1985][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.604.502 [graph_prepare.cc:1986][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.604.514 [graph_prepare.cc:1987][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.604.528 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.604.543 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.604.561 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.604.644 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.660 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.672 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.685 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.697 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.710 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.718 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.731 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.742 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.753 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.764 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [0] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.778 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.787 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.799 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.812 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.821 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.604.845 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.604.868 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.604.902 [graph_prepare.cc:1988][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [379] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.604.918 [graph_manager.cc:1065][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1077] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.616.931 [graph_manager.cc:1077][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11990] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.617.031 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.617.081 [graph_manager.cc:1080][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [112] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.620.462 [graph_manager.cc:1081][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3365] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.620.504 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.620.518 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.620.529 [graph_manager.cc:1082][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.620.559 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.620.573 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.620.587 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.620.619 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.620.633 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.620.646 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.620.659 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.620.696 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [28] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.620.714 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.620.732 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.620.757 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.620.782 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.620.795 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.620.805 [graph_manager.cc:2700][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [250] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.620.908 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.620.921 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.620.930 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.620.939 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.620.948 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.620.957 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CastRemovePass is [8] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.620.965 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.620.973 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.620.982 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.620.990 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.620.999 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [6] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.621.007 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.621.015 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.621.023 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.621.032 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.621.041 [graph_manager.cc:2741][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [220] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.050 [graph_manager.cc:2752][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.072 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.083 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.099 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.120 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.132 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.144 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.163 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.178 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.190 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.201 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.214 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.225 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.243 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.256 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.266 [graph_manager.cc:2810][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [197] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.292 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.621.303 [graph_manager.cc:2821][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [29] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.330 [graph_manager.cc:1087][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [783] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.461 [graph_manager.cc:1088][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [118] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.498 [graph_manager.cc:1089][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.516 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.530 [graph_manager.cc:1097][EVENT]167075 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:35.621.550 [graph_manager.cc:3325][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.928 [engine_place.cc:144][EVENT]167075 Run:The time cost of AIcoreEngine::CheckSupported is [282] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.957 [engine_place.cc:144][EVENT]167075 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.621.977 [engine_place.cc:144][EVENT]167075 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.622.050 [graph_manager.cc:3351][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [486] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.622.068 [graph_manager.cc:3364][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.622.129 [engine_partitioner.cc:1139][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.622.146 [engine_partitioner.cc:1142][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.622.270 [engine_partitioner.cc:1148][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [114] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.622.310 [engine_partitioner.cc:1155][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [27] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.622.353 [engine_partitioner.cc:1164][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.622.385 [graph_manager.cc:3405][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [304] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.622.403 [graph_manager.cc:3412][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.176 [graph_manager.cc:3422][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [91757] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.237 [graph_manager.cc:3428][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.395 [graph_manager.cc:3467][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [128] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.415 [graph_manager.cc:3377][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [92336] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.433 [graph_manager.cc:1106][EVENT]167075 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [92889] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.447 [graph_manager.cc:1115][EVENT]167075 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:35.714.470 [graph_manager.cc:1130][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.504 [graph_manager.cc:1131][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.530 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.548 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.573 [graph_manager.cc:2837][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [52] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.662 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.714.676 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.714.686 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.714.695 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.714.704 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [4] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.714.712 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [3] [INFO] GE(164043,python):2024-01-10-11:37:35.714.722 [graph_manager.cc:2864][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [130] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.734 [graph_manager.cc:2872][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.754 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.767 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.783 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.797 [compile_nodes_pass.cc:88][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.807 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.817 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.890 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [62] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.930 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [28] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.944 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.957 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.972 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.982 [graph_manager.cc:2927][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [232] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.714.994 [graph_manager.cc:2937][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.715.016 [graph_manager.cc:2943][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.715.028 [graph_manager.cc:2950][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.715.201 [graph_manager.cc:2958][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.715.232 [graph_manager.cc:1132][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [715] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.715.308 [graph_manager.cc:1135][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [63] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.715.339 [graph_manager.cc:2975][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.715.368 [graph_manager.cc:2981][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.715.383 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.715.393 [graph_manager.cc:2986][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.715.403 [graph_manager.cc:1136][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [80] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.715.527 [graph_manager.cc:3555][EVENT]167075 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [95] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.715.612 [engine_partitioner.cc:1139][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.715.628 [engine_partitioner.cc:1142][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.715.726 [engine_partitioner.cc:1148][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [88] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.715.759 [engine_partitioner.cc:1155][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.715.801 [engine_partitioner.cc:1164][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.715.823 [graph_builder.cc:865][EVENT]167075 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [244] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:35.716.106 [logger.cc:1071] 167075 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:37:35.716.135 [task_generator.cc:804][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [75] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.716.192 [task_generator.cc:805][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [44] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.717.047 [task_generator.cc:814][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [831] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.717.062 [task_generator.cc:954][EVENT]167075 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1002] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.717.126 [task_generator.cc:967][EVENT]167075 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [36] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:35.717.145 [logger.cc:1084] 167075 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:37:35.718.269 [graph_manager.cc:1152][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2842] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.718.308 [graph_manager.cc:1164][EVENT]167075 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:35.718.344 [graph_manager.cc:1271][EVENT]167075 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [118442] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.718.356 [graph_manager.cc:1272][EVENT]167075 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:37:35.718.685 [atrace_api.c:93](tid:167075) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:37:35.718.702 [atrace_api.c:95](tid:167075) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:37:35.740.965 [graph_converter.cc:838][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [7267] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.741.146 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [127] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.741.816 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [645] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.742.031 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [186] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.742.054 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [210] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.742.238 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [171] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.742.281 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.742.316 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.742.543 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [215] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.742.642 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [80] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.742.657 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [95] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.742.689 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.742.718 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.742.748 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.742.842 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [84] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.742.924 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [69] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.742.955 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [101] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.742.986 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.743.014 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.743.028 [graph_converter.cc:849][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [2015] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.743.307 [graph_converter.cc:853][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [270] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.744.146 [graph_converter.cc:857][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [823] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.744.314 [graph_converter.cc:862][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [141] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.828.688 [graph_var_manager.cc:1424][EVENT]167073 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:35.828.788 [graph_manager.cc:1248][EVENT]167073 PreRun:PreRun start: graph node size 4, session id 17, graph id 16, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:35.829.019 [atrace_api.c:28](tid:167073) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:35.829.051 [trace_rb_log.c:84](tid:167073) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:35.829.063 [atrace_api.c:32](tid:167073) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:35.829.080 [client_manager.cpp:157][SetProfilingCallback][tid:167073] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:35.829.451 [parallel_partitioner.cc:165][EVENT]167073 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.829.488 [parallel_partitioner.cc:178][EVENT]167073 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.829.533 [graph_prepare.cc:1378][EVENT]167073 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.829.669 [graph_manager.cc:1050][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [152] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.829.747 [graph_manager.cc:1052][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.829.888 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.829.919 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.829.965 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.829.979 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.830.049 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.830.063 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.830.081 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.830.181 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.830.202 [graph_manager.cc:1054][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [441] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.830.415 [graph_manager.cc:1055][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [200] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.831.436 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:35.831.467 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.831.479 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.831.489 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferShapePass is [324] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.831.499 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.831.508 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:35.831.516 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.831.525 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.831.536 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferValuePass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.833.566 [graph_manager.cc:1056][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3131] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.833.636 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.833.655 [graph_prepare.cc:1982][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.834.158 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:35.834.190 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.201 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.211 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferShapePass is [266] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.220 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.246 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:35.834.256 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [9] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.265 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.274 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.300 [graph_prepare.cc:1983][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [632] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.834.324 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.834.338 [graph_prepare.cc:1984][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [24] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.834.355 [graph_prepare.cc:1985][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.834.372 [graph_prepare.cc:1986][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.834.387 [graph_prepare.cc:1987][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.834.404 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.834.419 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.834.437 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.834.533 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.549 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.561 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.570 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.582 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.593 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.602 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.614 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.622 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.642 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.654 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.666 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.678 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.686 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.698 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.710 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.834.734 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.834.752 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.834.787 [graph_prepare.cc:1988][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [389] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.834.803 [graph_manager.cc:1065][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1202] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.847.231 [graph_manager.cc:1077][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12404] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.847.302 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.847.351 [graph_manager.cc:1080][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [82] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.855.897 [graph_manager.cc:1081][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [8529] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.855.945 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.855.960 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.855.972 [graph_manager.cc:1082][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.004 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.019 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.034 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.127 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [82] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.155 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.203 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.219 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.260 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.280 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.307 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.335 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.350 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.363 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.372 [graph_manager.cc:2700][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [374] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.499 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.856.512 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.856.522 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.856.531 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.856.539 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.856.548 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CastRemovePass is [9] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.856.557 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.856.565 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.856.573 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.856.582 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.856.590 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.856.598 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.856.607 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.856.624 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.856.633 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.856.643 [graph_manager.cc:2741][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [253] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.653 [graph_manager.cc:2752][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.676 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.687 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.705 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.720 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.732 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.746 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.764 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.778 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.792 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.802 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.815 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.826 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.844 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.857 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.865 [graph_manager.cc:2810][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [194] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.895 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.856.907 [graph_manager.cc:2821][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.856.935 [graph_manager.cc:1087][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [943] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.857.080 [graph_manager.cc:1088][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [120] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.857.124 [graph_manager.cc:1089][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.857.142 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.857.157 [graph_manager.cc:1097][EVENT]167073 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:35.857.178 [graph_manager.cc:3325][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.857.571 [engine_place.cc:144][EVENT]167073 Run:The time cost of AIcoreEngine::CheckSupported is [293] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.857.601 [engine_place.cc:144][EVENT]167073 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.857.611 [engine_place.cc:144][EVENT]167073 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.857.702 [graph_manager.cc:3351][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [510] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.857.724 [graph_manager.cc:3364][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.857.795 [engine_partitioner.cc:1139][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.857.813 [engine_partitioner.cc:1142][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.857.971 [engine_partitioner.cc:1148][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [148] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.858.014 [engine_partitioner.cc:1155][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.858.064 [engine_partitioner.cc:1164][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.858.098 [graph_manager.cc:3405][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [362] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.858.116 [graph_manager.cc:3412][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.003 [graph_manager.cc:3422][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [15872] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.043 [graph_manager.cc:3428][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.178 [graph_manager.cc:3467][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [114] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.196 [graph_manager.cc:3377][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [16462] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.224 [graph_manager.cc:1106][EVENT]167073 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [17052] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.237 [graph_manager.cc:1115][EVENT]167073 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:35.874.261 [graph_manager.cc:1130][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.293 [graph_manager.cc:1131][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.318 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.334 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.344 [graph_manager.cc:2837][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.427 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.874.440 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.874.449 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.874.458 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.874.466 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.874.475 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [7] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.874.485 [graph_manager.cc:2864][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [124] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.497 [graph_manager.cc:2872][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.515 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.529 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.545 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.559 [compile_nodes_pass.cc:88][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.569 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.579 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.674 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [72] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.703 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.716 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.729 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.743 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.753 [graph_manager.cc:2927][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [240] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.765 [graph_manager.cc:2937][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.779 [graph_manager.cc:2943][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.790 [graph_manager.cc:2950][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.963 [graph_manager.cc:2958][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [39] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.874.996 [graph_manager.cc:1132][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [689] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.875.064 [graph_manager.cc:1135][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [56] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.875.097 [graph_manager.cc:2975][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.875.127 [graph_manager.cc:2981][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.875.141 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.875.151 [graph_manager.cc:2986][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.875.160 [graph_manager.cc:1136][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [80] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.875.299 [graph_manager.cc:3555][EVENT]167073 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [104] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.875.391 [engine_partitioner.cc:1139][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.875.408 [engine_partitioner.cc:1142][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.875.529 [engine_partitioner.cc:1148][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [111] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.875.575 [engine_partitioner.cc:1155][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.875.618 [engine_partitioner.cc:1164][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [29] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.875.641 [graph_builder.cc:865][EVENT]167073 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [285] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:35.875.931 [logger.cc:1071] 167073 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:37:35.875.963 [task_generator.cc:804][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [73] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.876.026 [task_generator.cc:805][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [50] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.876.721 [task_generator.cc:814][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [681] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.876.737 [task_generator.cc:954][EVENT]167073 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [847] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.876.793 [task_generator.cc:967][EVENT]167073 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [32] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:35.876.812 [logger.cc:1084] 167073 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:37:35.877.409 [graph_manager.cc:1152][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2220] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.877.443 [graph_manager.cc:1164][EVENT]167073 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:35.877.479 [graph_manager.cc:1271][EVENT]167073 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [48116] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.877.490 [graph_manager.cc:1272][EVENT]167073 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:37:35.877.838 [atrace_api.c:93](tid:167073) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:37:35.877.859 [atrace_api.c:95](tid:167073) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:37:35.889.279 [graph_converter.cc:838][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [3742] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.889.446 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [118] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.889.955 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [484] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.890.172 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [187] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.890.194 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [211] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.890.413 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [207] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.890.455 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.890.487 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.890.702 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [193] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.890.790 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [69] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.890.805 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [85] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.890.835 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.890.860 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.890.887 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.890.968 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [70] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.891.037 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [57] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.891.049 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [70] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.891.075 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.891.100 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.891.114 [graph_converter.cc:849][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1792] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.891.341 [graph_converter.cc:853][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [218] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.892.059 [graph_converter.cc:857][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [702] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.892.211 [graph_converter.cc:862][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [126] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.961.093 [graph_var_manager.cc:1424][EVENT]167074 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:35.961.193 [graph_manager.cc:1248][EVENT]167074 PreRun:PreRun start: graph node size 4, session id 18, graph id 17, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:35.961.469 [atrace_api.c:28](tid:167074) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:35.961.504 [trace_rb_log.c:84](tid:167074) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:35.961.520 [atrace_api.c:32](tid:167074) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:35.961.536 [client_manager.cpp:157][SetProfilingCallback][tid:167074] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:35.961.970 [parallel_partitioner.cc:165][EVENT]167074 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.962.012 [parallel_partitioner.cc:178][EVENT]167074 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.962.095 [graph_prepare.cc:1378][EVENT]167074 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.962.252 [graph_manager.cc:1050][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [174] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.962.280 [graph_manager.cc:1052][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.962.415 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.962.447 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.962.496 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.962.509 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.962.552 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.962.566 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.962.583 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.962.681 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.962.701 [graph_manager.cc:1054][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [406] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.962.907 [graph_manager.cc:1055][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [193] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.963.924 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:35.963.955 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.963.967 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.963.977 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [353] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.963.986 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.963.994 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:35.964.003 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [10] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.964.012 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.964.021 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.966.076 [graph_manager.cc:1056][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3144] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.966.147 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.966.167 [graph_prepare.cc:1982][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [54] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.966.593 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:35.966.620 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.966.631 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.966.640 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [245] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.966.649 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.966.658 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:35.966.666 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.966.675 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.966.683 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.966.708 [graph_prepare.cc:1983][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [527] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.966.731 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.966.743 [graph_prepare.cc:1984][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.966.756 [graph_prepare.cc:1985][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.966.770 [graph_prepare.cc:1986][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.966.782 [graph_prepare.cc:1987][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.966.798 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.966.810 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.966.823 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.966.913 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.966.925 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.966.945 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.966.954 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.966.963 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.966.972 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.966.980 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.966.989 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.966.997 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.967.005 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.967.014 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.967.022 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SnapshotPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.967.030 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.967.038 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.967.047 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.967.055 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.967.078 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.967.091 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.967.123 [graph_prepare.cc:1988][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [332] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.967.136 [graph_manager.cc:1065][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1025] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.979.131 [graph_manager.cc:1077][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11975] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.979.201 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.979.249 [graph_manager.cc:1080][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [81] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.228 [graph_manager.cc:1081][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [4964] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.283 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.298 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.309 [graph_manager.cc:1082][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.340 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.354 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.368 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.399 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.413 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.428 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.440 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.480 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.498 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.517 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.543 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.558 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.570 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.580 [graph_manager.cc:2700][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [244] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.698 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of EnterPass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.984.711 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AddNPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.984.724 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.984.736 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.984.745 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.984.764 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CastRemovePass is [9] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.984.773 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.984.781 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.984.790 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.984.801 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.984.809 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.984.821 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.984.830 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.984.838 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.984.846 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.984.856 [graph_manager.cc:2741][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [258] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.865 [graph_manager.cc:2752][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.887 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.899 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.916 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.929 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.941 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.953 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.971 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.985 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.984.997 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.985.007 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.985.019 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.985.035 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.985.053 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.985.066 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.985.074 [graph_manager.cc:2810][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [191] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.985.103 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:35.985.117 [graph_manager.cc:2821][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.985.144 [graph_manager.cc:1087][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [816] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.985.279 [graph_manager.cc:1088][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [122] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.985.321 [graph_manager.cc:1089][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.985.338 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.985.353 [graph_manager.cc:1097][EVENT]167074 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:35.985.373 [graph_manager.cc:3325][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.985.950 [engine_place.cc:144][EVENT]167074 Run:The time cost of AIcoreEngine::CheckSupported is [473] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.985.981 [engine_place.cc:144][EVENT]167074 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.985.991 [engine_place.cc:144][EVENT]167074 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.986.066 [graph_manager.cc:3351][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [680] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.986.089 [graph_manager.cc:3364][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.986.154 [engine_partitioner.cc:1139][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.986.172 [engine_partitioner.cc:1142][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.986.325 [engine_partitioner.cc:1148][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [143] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.986.372 [engine_partitioner.cc:1155][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [31] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.986.429 [engine_partitioner.cc:1164][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.986.463 [graph_manager.cc:3405][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [361] micro second. [INFO] GE(164043,python):2024-01-10-11:37:35.986.480 [graph_manager.cc:3412][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.086 [graph_manager.cc:3422][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [19591] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.139 [graph_manager.cc:3428][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.288 [graph_manager.cc:3467][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [126] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.307 [graph_manager.cc:3377][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [20207] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.323 [graph_manager.cc:1106][EVENT]167074 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [20957] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.336 [graph_manager.cc:1115][EVENT]167074 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:36.006.360 [graph_manager.cc:1130][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.392 [graph_manager.cc:1131][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.418 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.433 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.443 [graph_manager.cc:2837][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.523 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.006.536 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.006.546 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.006.555 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of BitcastPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.006.564 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.006.572 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.006.582 [graph_manager.cc:2864][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [123] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.606 [graph_manager.cc:2872][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.626 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.641 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.657 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.670 [compile_nodes_pass.cc:88][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.681 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.691 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.771 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [71] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.799 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.812 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.826 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.839 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.848 [graph_manager.cc:2927][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [224] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.861 [graph_manager.cc:2937][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.875 [graph_manager.cc:2943][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.006.886 [graph_manager.cc:2950][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.007.060 [graph_manager.cc:2958][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.007.091 [graph_manager.cc:1132][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [684] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.007.160 [graph_manager.cc:1135][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [57] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.007.191 [graph_manager.cc:2975][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.007.222 [graph_manager.cc:2981][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.007.245 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.007.255 [graph_manager.cc:2986][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.007.264 [graph_manager.cc:1136][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [88] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.007.379 [graph_manager.cc:3555][EVENT]167074 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [83] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.007.470 [engine_partitioner.cc:1139][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.007.486 [engine_partitioner.cc:1142][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.007.606 [engine_partitioner.cc:1148][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [110] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.007.639 [engine_partitioner.cc:1155][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.007.680 [engine_partitioner.cc:1164][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [28] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.007.702 [graph_builder.cc:865][EVENT]167074 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [265] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:36.007.995 [logger.cc:1071] 167074 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:37:36.008.026 [task_generator.cc:804][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [82] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.008.086 [task_generator.cc:805][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [48] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.008.916 [task_generator.cc:814][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [814] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.008.935 [task_generator.cc:954][EVENT]167074 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [992] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.008.994 [task_generator.cc:967][EVENT]167074 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [32] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:36.009.013 [logger.cc:1084] 167074 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:37:36.009.921 [graph_manager.cc:1152][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2630] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.009.955 [graph_manager.cc:1164][EVENT]167074 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:36.009.990 [graph_manager.cc:1271][EVENT]167074 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [48102] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.010.000 [graph_manager.cc:1272][EVENT]167074 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:37:36.010.314 [atrace_api.c:93](tid:167074) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:37:36.010.343 [atrace_api.c:95](tid:167074) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:37:36.023.700 [graph_converter.cc:838][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [3922] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.023.872 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [121] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.024.386 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [489] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.024.603 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [192] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.024.625 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [215] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.024.843 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [206] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.024.883 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.024.913 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.025.115 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [190] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.025.201 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [67] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.025.215 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [82] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.025.244 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.025.270 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.025.296 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.025.376 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [69] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.025.444 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [57] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.025.455 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [68] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.025.481 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.025.506 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.025.520 [graph_converter.cc:849][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1777] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.025.789 [graph_converter.cc:853][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [259] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.026.517 [graph_converter.cc:857][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [709] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.026.667 [graph_converter.cc:862][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [123] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.096.547 [graph_var_manager.cc:1424][EVENT]167073 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:36.096.648 [graph_manager.cc:1248][EVENT]167073 PreRun:PreRun start: graph node size 4, session id 19, graph id 18, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:36.096.877 [atrace_api.c:28](tid:167073) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:36.096.909 [trace_rb_log.c:84](tid:167073) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:36.096.922 [atrace_api.c:32](tid:167073) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:36.096.939 [client_manager.cpp:157][SetProfilingCallback][tid:167073] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:36.097.289 [parallel_partitioner.cc:165][EVENT]167073 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.097.329 [parallel_partitioner.cc:178][EVENT]167073 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.097.374 [graph_prepare.cc:1378][EVENT]167073 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.097.500 [graph_manager.cc:1050][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [141] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.097.524 [graph_manager.cc:1052][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.097.657 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.097.739 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [54] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.097.794 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.097.812 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.097.861 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.097.878 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.097.897 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.098.000 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.098.026 [graph_manager.cc:1054][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [490] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.098.239 [graph_manager.cc:1055][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [195] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.099.251 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:36.099.282 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.099.319 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.099.332 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferShapePass is [318] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.099.344 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.099.355 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:36.099.364 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.099.376 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.099.386 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferValuePass is [9] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.101.391 [graph_manager.cc:1056][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3128] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.101.462 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.101.485 [graph_prepare.cc:1982][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [57] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.101.966 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:36.101.994 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.005 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.014 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferShapePass is [282] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.023 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.032 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:36.102.040 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.049 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.057 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.082 [graph_prepare.cc:1983][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [584] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.102.106 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.102.119 [graph_prepare.cc:1984][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.102.133 [graph_prepare.cc:1985][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.102.159 [graph_prepare.cc:1986][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.102.171 [graph_prepare.cc:1987][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.102.188 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.102.203 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.102.217 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.102.307 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.319 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.328 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.337 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.346 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.354 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.363 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.371 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.379 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.388 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.396 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.404 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.413 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.421 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.429 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.438 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.102.460 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.102.474 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.102.514 [graph_prepare.cc:1988][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [331] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.102.527 [graph_manager.cc:1065][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1100] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.114.830 [graph_manager.cc:1077][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12283] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.114.901 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.114.950 [graph_manager.cc:1080][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [80] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.223 [graph_manager.cc:1081][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [8256] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.271 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.285 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.297 [graph_manager.cc:1082][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.330 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.344 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.359 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.452 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [82] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.470 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.515 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.530 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.571 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.591 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.617 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.644 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.659 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.684 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.694 [graph_manager.cc:2700][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [370] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.820 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.123.835 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.123.844 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.123.853 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.123.862 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.123.871 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.123.879 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.123.888 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.123.896 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.123.905 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.123.913 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.123.921 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.123.930 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.123.938 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.123.946 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.123.956 [graph_manager.cc:2741][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [243] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.965 [graph_manager.cc:2752][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.987 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.123.998 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.016 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.031 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.051 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.065 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.084 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.099 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.112 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.122 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.134 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.145 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.164 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.178 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.188 [graph_manager.cc:2810][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [205] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.219 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.124.231 [graph_manager.cc:2821][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.258 [graph_manager.cc:1087][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [941] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.393 [graph_manager.cc:1088][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [123] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.434 [graph_manager.cc:1089][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.451 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.467 [graph_manager.cc:1097][EVENT]167073 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:36.124.488 [graph_manager.cc:3325][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.879 [engine_place.cc:144][EVENT]167073 Run:The time cost of AIcoreEngine::CheckSupported is [292] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.907 [engine_place.cc:144][EVENT]167073 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.124.917 [engine_place.cc:144][EVENT]167073 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.125.009 [graph_manager.cc:3351][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [508] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.125.029 [graph_manager.cc:3364][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.125.095 [engine_partitioner.cc:1139][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.125.113 [engine_partitioner.cc:1142][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.125.266 [engine_partitioner.cc:1148][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [143] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.125.311 [engine_partitioner.cc:1155][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [31] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.125.359 [engine_partitioner.cc:1164][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.125.391 [graph_manager.cc:3405][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [349] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.125.410 [graph_manager.cc:3412][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.141.539 [graph_manager.cc:3422][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [16115] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.141.588 [graph_manager.cc:3428][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.141.752 [graph_manager.cc:3467][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [143] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.141.774 [graph_manager.cc:3377][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [16733] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.141.791 [graph_manager.cc:1106][EVENT]167073 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [17310] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.141.804 [graph_manager.cc:1115][EVENT]167073 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:36.141.827 [graph_manager.cc:1130][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.141.859 [graph_manager.cc:1131][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.141.885 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.141.903 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.141.913 [graph_manager.cc:2837][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.017 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.142.030 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.142.039 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.142.048 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.142.057 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.142.066 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [9] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.142.075 [graph_manager.cc:2864][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [134] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.088 [graph_manager.cc:2872][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.107 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.122 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.138 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.152 [compile_nodes_pass.cc:88][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.162 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.173 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.256 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [73] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.287 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.301 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.315 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.327 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.336 [graph_manager.cc:2927][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [233] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.349 [graph_manager.cc:2937][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.371 [graph_manager.cc:2943][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.382 [graph_manager.cc:2950][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.558 [graph_manager.cc:2958][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [40] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.596 [graph_manager.cc:1132][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [721] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.673 [graph_manager.cc:1135][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [60] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.705 [graph_manager.cc:2975][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.736 [graph_manager.cc:2981][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.750 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.760 [graph_manager.cc:2986][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.769 [graph_manager.cc:1136][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [80] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.142.910 [graph_manager.cc:3555][EVENT]167073 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [108] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.143.007 [engine_partitioner.cc:1139][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.143.027 [engine_partitioner.cc:1142][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.143.155 [engine_partitioner.cc:1148][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [117] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.143.191 [engine_partitioner.cc:1155][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.143.233 [engine_partitioner.cc:1164][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [31] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.143.257 [graph_builder.cc:865][EVENT]167073 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [286] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:36.143.568 [logger.cc:1071] 167073 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:37:36.143.600 [task_generator.cc:804][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [82] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.143.663 [task_generator.cc:805][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [51] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.144.366 [task_generator.cc:814][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [687] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.144.391 [task_generator.cc:954][EVENT]167073 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [874] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.144.450 [task_generator.cc:967][EVENT]167073 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [32] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:36.144.469 [logger.cc:1084] 167073 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:37:36.145.071 [graph_manager.cc:1152][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2275] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.145.105 [graph_manager.cc:1164][EVENT]167073 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:36.145.140 [graph_manager.cc:1271][EVENT]167073 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [47938] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.145.153 [graph_manager.cc:1272][EVENT]167073 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:37:36.145.470 [atrace_api.c:93](tid:167073) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:37:36.145.488 [atrace_api.c:95](tid:167073) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:37:36.157.264 [graph_converter.cc:838][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [3770] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.157.434 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [120] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.157.985 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [524] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.158.201 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [189] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.158.223 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [212] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.158.441 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [206] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.158.482 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.158.513 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.158.713 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [189] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.158.801 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [68] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.158.817 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [85] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.158.846 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.158.872 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.158.898 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.158.977 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [69] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.159.047 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [57] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.159.070 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [81] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.159.097 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.159.122 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.159.136 [graph_converter.cc:849][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1830] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.159.364 [graph_converter.cc:853][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [218] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.160.095 [graph_converter.cc:857][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [715] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.160.247 [graph_converter.cc:862][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [125] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.227.516 [graph_var_manager.cc:1424][EVENT]167073 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:36.227.615 [graph_manager.cc:1248][EVENT]167073 PreRun:PreRun start: graph node size 4, session id 20, graph id 19, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:36.227.847 [atrace_api.c:28](tid:167073) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:36.227.880 [trace_rb_log.c:84](tid:167073) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:36.227.893 [atrace_api.c:32](tid:167073) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:36.227.909 [client_manager.cpp:157][SetProfilingCallback][tid:167073] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:36.228.264 [parallel_partitioner.cc:165][EVENT]167073 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.228.300 [parallel_partitioner.cc:178][EVENT]167073 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.228.346 [graph_prepare.cc:1378][EVENT]167073 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.228.521 [graph_manager.cc:1050][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [191] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.228.545 [graph_manager.cc:1052][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.228.680 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.228.711 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.228.759 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.228.773 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.228.817 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.228.856 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.228.875 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.228.976 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.228.996 [graph_manager.cc:1054][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [439] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.229.206 [graph_manager.cc:1055][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [196] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.230.341 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:36.230.373 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.230.384 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.230.394 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferShapePass is [436] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.230.403 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.230.412 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:36.230.421 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.230.430 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.230.438 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.233.238 [graph_manager.cc:1056][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [4011] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.233.307 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.233.326 [graph_prepare.cc:1982][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.233.881 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:36.233.912 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.233.923 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.233.932 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferShapePass is [352] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.233.941 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.233.962 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:37:36.233.971 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.233.980 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.233.988 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.234.015 [graph_prepare.cc:1983][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [675] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.234.041 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.234.055 [graph_prepare.cc:1984][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.234.069 [graph_prepare.cc:1985][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.234.084 [graph_prepare.cc:1986][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.234.098 [graph_prepare.cc:1987][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.234.114 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.234.126 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.234.140 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.234.231 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.234.246 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.234.257 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.234.266 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.234.274 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.234.283 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.234.291 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.234.300 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.234.308 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.234.316 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.234.333 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.234.342 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.234.350 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.234.359 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.234.367 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.234.377 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:37:36.234.400 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.234.416 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.234.449 [graph_prepare.cc:1988][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [340] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.234.461 [graph_manager.cc:1065][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1189] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.247.494 [graph_manager.cc:1077][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13011] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.247.592 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.247.648 [graph_manager.cc:1080][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [114] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.258.591 [graph_manager.cc:1081][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [10924] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.258.638 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.258.653 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.258.665 [graph_manager.cc:1082][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.258.698 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.258.714 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.258.730 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.258.877 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [136] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.258.893 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.258.998 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [82] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.014 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.064 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.084 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.105 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.191 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [75] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.208 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.222 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.232 [graph_manager.cc:2700][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [540] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.471 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of EnterPass is [6] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.259.487 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.259.497 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [7] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.259.506 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.259.515 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [7] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.259.524 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CastRemovePass is [39] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.259.533 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.259.541 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.259.550 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [10] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.259.558 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.259.566 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [17] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.259.575 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.259.583 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.259.601 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [9] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.259.610 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [4] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.259.620 [graph_manager.cc:2741][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [368] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.630 [graph_manager.cc:2752][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.654 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.666 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.687 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.703 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.716 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.729 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.750 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.766 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.780 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.790 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.803 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.815 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.837 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.851 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.861 [graph_manager.cc:2810][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [212] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.905 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of IdentityPass is [5] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.259.917 [graph_manager.cc:2821][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [47] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.259.944 [graph_manager.cc:1087][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1261] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.260.498 [graph_manager.cc:1088][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [539] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.260.572 [graph_manager.cc:1089][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.260.596 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.260.614 [graph_manager.cc:1097][EVENT]167073 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:36.260.638 [graph_manager.cc:3325][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.270.307 [engine_place.cc:144][EVENT]167073 Run:The time cost of AIcoreEngine::CheckSupported is [9449] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.270.342 [engine_place.cc:144][EVENT]167073 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.270.352 [engine_place.cc:144][EVENT]167073 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.270.444 [graph_manager.cc:3351][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [9793] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.270.463 [graph_manager.cc:3364][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.270.540 [engine_partitioner.cc:1139][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [27] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.270.571 [engine_partitioner.cc:1142][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.270.738 [engine_partitioner.cc:1148][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [156] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.270.779 [engine_partitioner.cc:1155][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [28] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.270.828 [engine_partitioner.cc:1164][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.270.863 [graph_manager.cc:3405][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [386] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.270.882 [graph_manager.cc:3412][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.095 [graph_manager.cc:3422][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [115199] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.144 [graph_manager.cc:3428][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.310 [graph_manager.cc:3467][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [146] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.329 [graph_manager.cc:3377][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [115853] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.360 [graph_manager.cc:1106][EVENT]167073 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [125730] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.373 [graph_manager.cc:1115][EVENT]167073 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:36.386.398 [graph_manager.cc:1130][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.432 [graph_manager.cc:1131][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.461 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.480 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.490 [graph_manager.cc:2837][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [42] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.624 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [24] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.386.638 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.386.647 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondRemovePass is [8] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.386.656 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of BitcastPass is [3] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.386.665 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [9] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.386.674 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [14] micro second, call num is [9] [INFO] GE(164043,python):2024-01-10-11:37:36.386.684 [graph_manager.cc:2864][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [175] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.697 [graph_manager.cc:2872][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.717 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.731 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.748 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.762 [compile_nodes_pass.cc:88][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.773 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.783 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.901 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [100] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.950 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.964 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.979 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.386.994 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.387.004 [graph_manager.cc:2927][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [291] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.387.016 [graph_manager.cc:2937][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.387.032 [graph_manager.cc:2943][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.387.044 [graph_manager.cc:2950][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.387.234 [graph_manager.cc:2958][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [55] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.387.267 [graph_manager.cc:1132][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [822] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.387.345 [graph_manager.cc:1135][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [65] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.387.381 [graph_manager.cc:2975][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.387.414 [graph_manager.cc:2981][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.387.428 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.387.438 [graph_manager.cc:2986][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.387.447 [graph_manager.cc:1136][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [86] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.387.778 [graph_manager.cc:3555][EVENT]167073 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [291] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.387.908 [engine_partitioner.cc:1139][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.387.938 [engine_partitioner.cc:1142][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.388.083 [engine_partitioner.cc:1148][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [134] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.388.130 [engine_partitioner.cc:1155][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.388.174 [engine_partitioner.cc:1164][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.388.201 [graph_builder.cc:865][EVENT]167073 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [356] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:36.388.589 [logger.cc:1071] 167073 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:37:36.388.623 [task_generator.cc:804][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [86] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.388.707 [task_generator.cc:805][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [71] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.390.378 [task_generator.cc:814][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1656] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.390.396 [task_generator.cc:954][EVENT]167073 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1859] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.390.467 [task_generator.cc:967][EVENT]167073 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [37] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:37:36.390.492 [logger.cc:1084] 167073 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:37:36.391.740 [graph_manager.cc:1152][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4259] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.391.779 [graph_manager.cc:1164][EVENT]167073 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:37:36.391.817 [graph_manager.cc:1271][EVENT]167073 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [163638] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.391.828 [graph_manager.cc:1272][EVENT]167073 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:37:36.392.150 [atrace_api.c:93](tid:167073) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:37:36.392.169 [atrace_api.c:95](tid:167073) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:37:36.421.822 [graph_converter.cc:838][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [11011] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.422.045 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [164] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.423.589 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [1514] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.423.985 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [363] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.424.013 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [394] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.424.277 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [249] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.424.361 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [58] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.424.431 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [50] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.424.910 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [447] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.425.128 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [189] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.425.152 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [214] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.425.217 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [53] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.425.275 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [44] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.425.334 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [47] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.425.559 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [212] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.425.788 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [212] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.425.811 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [235] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.425.873 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [52] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.425.931 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [44] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.425.950 [graph_converter.cc:849][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4075] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.426.657 [graph_converter.cc:853][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [696] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.428.591 [graph_converter.cc:857][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1907] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.428.992 [graph_converter.cc:862][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [369] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.517.469 [graph_var_manager.cc:1424][EVENT]167075 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:37:36.517.566 [graph_manager.cc:1248][EVENT]167075 PreRun:PreRun start: graph node size 5, session id 21, graph id 20, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:37:36.517.854 [atrace_api.c:28](tid:167075) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:37:36.517.886 [trace_rb_log.c:84](tid:167075) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:37:36.517.899 [atrace_api.c:32](tid:167075) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:37:36.517.914 [client_manager.cpp:157][SetProfilingCallback][tid:167075] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:37:36.518.286 [parallel_partitioner.cc:165][EVENT]167075 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.518.325 [parallel_partitioner.cc:178][EVENT]167075 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.518.398 [graph_prepare.cc:1378][EVENT]167075 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.518.525 [graph_manager.cc:1050][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [144] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.518.548 [graph_manager.cc:1052][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.518.692 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.518.722 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.518.769 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.518.782 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.518.825 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.518.839 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.518.858 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.518.960 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.518.980 [graph_manager.cc:1054][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [420] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.519.190 [graph_manager.cc:1055][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [197] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.520.360 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:36.520.390 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.520.402 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.520.412 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [400] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.520.421 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.520.430 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:36.520.439 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.520.447 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.520.456 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.522.790 [graph_manager.cc:1056][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3580] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.522.874 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.522.894 [graph_prepare.cc:1982][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [56] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.523.485 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:36.523.512 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.523 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.533 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [333] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.542 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.550 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:37:36.523.559 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.567 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.576 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.628 [graph_prepare.cc:1983][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [720] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.523.653 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.523.665 [graph_prepare.cc:1984][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.523.679 [graph_prepare.cc:1985][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.523.693 [graph_prepare.cc:1986][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.523.706 [graph_prepare.cc:1987][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.523.720 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.523.732 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.523.747 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.523.849 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.861 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.881 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.890 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.899 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.907 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.916 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.924 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [3] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.932 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.941 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [3] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.949 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.957 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.966 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.974 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [8] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.982 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.523.990 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:37:36.524.013 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.524.026 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.524.060 [graph_prepare.cc:1988][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [346] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.524.073 [graph_manager.cc:1065][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1235] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.536.247 [graph_manager.cc:1077][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12154] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.536.354 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.536.407 [graph_manager.cc:1080][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [121] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.545.092 [graph_manager.cc:1081][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [8667] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.545.152 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.545.169 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.545.181 [graph_manager.cc:1082][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.545.215 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.545.230 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.545.245 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.545.426 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [169] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.545.443 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.545.564 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [110] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.545.581 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.545.636 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [44] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.545.657 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.545.679 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.545.797 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [97] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.545.816 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.545.830 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.545.840 [graph_manager.cc:2700][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [632] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.546.130 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:37:36.546.146 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:37:36.546.157 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:37:36.546.166 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:37:36.546.175 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:37:36.546.184 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CastRemovePass is [51] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:37:36.546.203 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:37:36.546.213 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:37:36.546.221 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [12] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:37:36.546.230 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [4] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:37:36.546.239 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [20] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:37:36.546.247 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:37:36.546.256 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [21] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:37:36.546.264 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [11] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:37:36.546.272 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [5] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:37:36.546.282 [graph_manager.cc:2741][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [423] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.546.291 [graph_manager.cc:2752][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.546.315 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.546.328 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.546.353 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.546.369 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.546.381 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.546.395 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.546.416 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.546.431 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.546.444 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.546.454 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.546.466 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.546.484 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.546.509 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.546.523 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.546.532 [graph_manager.cc:2810][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [222] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.546.584 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:37:36.546.597 [graph_manager.cc:2821][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [55] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.546.624 [graph_manager.cc:1087][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1422] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.547.306 [graph_manager.cc:1088][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [667] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.547.374 [graph_manager.cc:1089][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.547.398 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.547.416 [graph_manager.cc:1097][EVENT]167075 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:37:36.547.439 [graph_manager.cc:3325][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.558.207 [engine_place.cc:144][EVENT]167075 Run:The time cost of AIcoreEngine::CheckSupported is [10495] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.558.241 [engine_place.cc:144][EVENT]167075 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.558.252 [engine_place.cc:144][EVENT]167075 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.558.352 [graph_manager.cc:3351][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10899] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.558.372 [graph_manager.cc:3364][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.558.458 [engine_partitioner.cc:1139][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.558.493 [engine_partitioner.cc:1142][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.558.697 [engine_partitioner.cc:1148][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [193] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.558.745 [engine_partitioner.cc:1155][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.558.795 [engine_partitioner.cc:1164][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.558.843 [graph_manager.cc:3405][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [458] micro second. [INFO] GE(164043,python):2024-01-10-11:37:36.558.862 [graph_manager.cc:3412][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. - \ | / - \ | [INFO] GE(164043,python):2024-01-10-11:38:47.578.014 [graph_manager.cc:3422][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [71019136] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.578.112 [graph_manager.cc:3428][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.578.382 [graph_manager.cc:3467][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [238] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.578.405 [graph_manager.cc:3377][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [71020021] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.578.424 [graph_manager.cc:1106][EVENT]167075 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [71030993] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.578.439 [graph_manager.cc:1115][EVENT]167075 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:38:47.578.475 [graph_manager.cc:1130][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.578.516 [graph_manager.cc:1131][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.578.549 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.578.573 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.578.584 [graph_manager.cc:2837][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [52] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.578.778 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [40] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:47.578.793 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:47.578.803 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:47.578.812 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:47.578.822 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [11] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:47.578.831 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [22] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:47.578.843 [graph_manager.cc:2864][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [236] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.578.874 [graph_manager.cc:2872][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.578.897 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.578.913 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.578.932 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.578.948 [compile_nodes_pass.cc:88][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.578.958 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.578.969 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.579.105 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [126] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.579.212 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [93] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.579.227 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.579.243 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.579.260 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.579.269 [graph_manager.cc:2927][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [377] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.579.283 [graph_manager.cc:2937][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.579.299 [graph_manager.cc:2943][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.579.310 [graph_manager.cc:2950][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.579.645 [graph_manager.cc:2958][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [67] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.579.682 [graph_manager.cc:1132][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [1152] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.579.800 [graph_manager.cc:1135][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [105] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.579.852 [graph_manager.cc:2975][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.579.889 [graph_manager.cc:2981][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.579.915 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.579.925 [graph_manager.cc:2986][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.579.935 [graph_manager.cc:1136][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [116] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.580.355 [graph_manager.cc:3555][EVENT]167075 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [369] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.580.520 [engine_partitioner.cc:1139][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [45] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.580.560 [engine_partitioner.cc:1142][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.580.766 [engine_partitioner.cc:1148][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [194] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.580.810 [engine_partitioner.cc:1155][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [31] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.580.861 [engine_partitioner.cc:1164][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.580.892 [graph_builder.cc:865][EVENT]167075 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [454] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:38:47.581.464 [logger.cc:1071] 167075 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:38:47.581.502 [task_generator.cc:804][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [155] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.581.613 [task_generator.cc:805][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [96] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.583.813 [task_generator.cc:814][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [2185] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.583.830 [task_generator.cc:954][EVENT]167075 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2483] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.583.906 [task_generator.cc:967][EVENT]167075 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [40] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:38:47.583.931 [logger.cc:1084] 167075 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:38:47.586.129 [graph_manager.cc:1152][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [6154] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.586.171 [graph_manager.cc:1164][EVENT]167075 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:38:47.586.214 [graph_manager.cc:1271][EVENT]167075 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [71068017] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.586.225 [graph_manager.cc:1272][EVENT]167075 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:38:47.586.634 [atrace_api.c:93](tid:167075) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:38:47.586.669 [atrace_api.c:95](tid:167075) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:38:47.616.157 [graph_converter.cc:838][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [11843] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.616.408 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [194] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.618.524 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [2087] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.619.051 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [494] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.619.080 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [525] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.619.388 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [293] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.619.491 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [77] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.619.575 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [62] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.620.163 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [571] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.620.427 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [234] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.620.452 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [261] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.620.531 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [67] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.620.605 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [58] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.620.678 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [58] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.620.964 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [272] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.621.208 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [221] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.621.230 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [244] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.621.306 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [65] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.621.378 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [57] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.621.400 [graph_converter.cc:849][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [5193] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.622.362 [graph_converter.cc:853][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [951] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.624.772 [graph_converter.cc:857][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2381] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.625.280 [graph_converter.cc:862][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [476] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.714.819 [graph_var_manager.cc:1424][EVENT]167073 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:38:47.714.919 [graph_manager.cc:1248][EVENT]167073 PreRun:PreRun start: graph node size 4, session id 22, graph id 21, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:38:47.715.183 [atrace_api.c:28](tid:167073) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:38:47.715.213 [trace_rb_log.c:84](tid:167073) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:38:47.715.227 [atrace_api.c:32](tid:167073) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:38:47.715.250 [client_manager.cpp:157][SetProfilingCallback][tid:167073] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:38:47.715.641 [parallel_partitioner.cc:165][EVENT]167073 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.715.680 [parallel_partitioner.cc:178][EVENT]167073 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.715.728 [graph_prepare.cc:1378][EVENT]167073 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.715.871 [graph_manager.cc:1050][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [160] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.715.894 [graph_manager.cc:1052][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.716.030 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.716.059 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.716.107 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.716.120 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.716.167 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.716.180 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.716.196 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.716.307 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.716.328 [graph_manager.cc:1054][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [422] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.716.544 [graph_manager.cc:1055][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [203] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.717.565 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:38:47.717.594 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.717.631 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.717.641 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferShapePass is [340] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.717.651 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.717.660 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:38:47.717.669 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.717.677 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.717.714 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.719.879 [graph_manager.cc:1056][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3315] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.719.948 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.719.966 [graph_prepare.cc:1982][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.720.461 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:38:47.720.488 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.499 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.509 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferShapePass is [266] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.518 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.526 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:38:47.720.535 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.543 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.551 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.599 [graph_prepare.cc:1983][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [619] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.720.626 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.720.638 [graph_prepare.cc:1984][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.720.653 [graph_prepare.cc:1985][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.720.683 [graph_prepare.cc:1986][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.720.694 [graph_prepare.cc:1987][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.720.710 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.720.722 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.720.737 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.720.828 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.841 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.850 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrintOpPass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.858 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.867 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.875 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.884 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.892 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.900 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.909 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.917 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.925 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.934 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.942 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.950 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.959 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.720.981 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.720.999 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.721.032 [graph_prepare.cc:1988][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [328] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.721.045 [graph_manager.cc:1065][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1132] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.734.409 [graph_manager.cc:1077][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13346] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.734.483 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.734.532 [graph_manager.cc:1080][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [86] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.739.713 [graph_manager.cc:1081][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [5165] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.739.757 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.739.770 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.739.782 [graph_manager.cc:1082][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.739.812 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.739.826 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.739.841 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.739.880 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [29] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.739.894 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.739.909 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.739.922 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.739.963 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.739.981 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.000 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.030 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.045 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.069 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.079 [graph_manager.cc:2700][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [271] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.198 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.740.211 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.740.220 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.740.229 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.740.238 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.740.247 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.740.255 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.740.263 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.740.272 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.740.280 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.740.288 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.740.296 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.740.305 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.740.313 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.740.321 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.740.331 [graph_manager.cc:2741][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [234] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.340 [graph_manager.cc:2752][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.363 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.374 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.391 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.406 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.425 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.438 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.457 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.472 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.485 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.497 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.511 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.523 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.541 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.555 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.564 [graph_manager.cc:2810][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [205] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.593 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.740.605 [graph_manager.cc:2821][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.631 [graph_manager.cc:1087][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [832] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.765 [graph_manager.cc:1088][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [121] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.805 [graph_manager.cc:1089][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.822 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.740.838 [graph_manager.cc:1097][EVENT]167073 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:38:47.740.859 [graph_manager.cc:3325][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.741.382 [engine_place.cc:144][EVENT]167073 Run:The time cost of AIcoreEngine::CheckSupported is [403] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.741.410 [engine_place.cc:144][EVENT]167073 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.741.420 [engine_place.cc:144][EVENT]167073 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.741.508 [graph_manager.cc:3351][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [637] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.741.527 [graph_manager.cc:3364][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.741.597 [engine_partitioner.cc:1139][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.741.615 [engine_partitioner.cc:1142][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.741.793 [engine_partitioner.cc:1148][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [167] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.741.840 [engine_partitioner.cc:1155][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [31] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.741.887 [engine_partitioner.cc:1164][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.741.923 [graph_manager.cc:3405][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [383] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.741.942 [graph_manager.cc:3412][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.225 [graph_manager.cc:3422][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [17268] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.269 [graph_manager.cc:3428][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.411 [graph_manager.cc:3467][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [122] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.430 [graph_manager.cc:3377][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [17891] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.446 [graph_manager.cc:1106][EVENT]167073 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [18594] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.459 [graph_manager.cc:1115][EVENT]167073 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:38:47.759.482 [graph_manager.cc:1130][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.514 [graph_manager.cc:1131][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.539 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.556 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.566 [graph_manager.cc:2837][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.663 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.759.676 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.759.685 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.759.694 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.759.703 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.759.712 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.759.721 [graph_manager.cc:2864][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [128] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.732 [graph_manager.cc:2872][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.752 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.766 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.781 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.795 [compile_nodes_pass.cc:88][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.805 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.815 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.895 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [70] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.925 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.938 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.951 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.965 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.974 [graph_manager.cc:2927][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [226] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.759.987 [graph_manager.cc:2937][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.760.009 [graph_manager.cc:2943][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.760.021 [graph_manager.cc:2950][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.760.188 [graph_manager.cc:2958][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.760.219 [graph_manager.cc:1132][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [691] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.760.292 [graph_manager.cc:1135][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [61] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.760.324 [graph_manager.cc:2975][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.760.359 [graph_manager.cc:2981][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.760.373 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.760.384 [graph_manager.cc:2986][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.760.393 [graph_manager.cc:1136][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [85] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.760.508 [graph_manager.cc:3555][EVENT]167073 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [83] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.760.599 [engine_partitioner.cc:1139][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.760.616 [engine_partitioner.cc:1142][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.760.737 [engine_partitioner.cc:1148][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [111] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.760.771 [engine_partitioner.cc:1155][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.760.811 [engine_partitioner.cc:1164][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [29] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.760.835 [graph_builder.cc:865][EVENT]167073 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [271] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:38:47.761.123 [logger.cc:1071] 167073 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:38:47.761.155 [task_generator.cc:804][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [85] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.761.218 [task_generator.cc:805][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [50] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.762.038 [task_generator.cc:814][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [806] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.762.064 [task_generator.cc:954][EVENT]167073 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [994] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.762.121 [task_generator.cc:967][EVENT]167073 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [32] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:38:47.762.139 [logger.cc:1084] 167073 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:38:47.762.959 [graph_manager.cc:1152][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2539] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.762.992 [graph_manager.cc:1164][EVENT]167073 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:38:47.763.027 [graph_manager.cc:1271][EVENT]167073 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [47477] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.763.037 [graph_manager.cc:1272][EVENT]167073 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:38:47.763.358 [atrace_api.c:93](tid:167073) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:38:47.763.375 [atrace_api.c:95](tid:167073) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:38:47.777.235 [graph_converter.cc:838][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [4691] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.777.406 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [123] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.777.955 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [525] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.778.172 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [188] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.778.194 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [212] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.778.412 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [205] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.778.456 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [24] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.778.488 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.778.688 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [188] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.778.777 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [71] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.778.791 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [85] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.778.821 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.778.846 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.778.873 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.778.952 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [68] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.779.021 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [58] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.779.046 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [82] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.779.074 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.779.098 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.779.112 [graph_converter.cc:849][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1833] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.779.341 [graph_converter.cc:853][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [220] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.780.071 [graph_converter.cc:857][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [714] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.780.224 [graph_converter.cc:862][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [126] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.857.781 [graph_var_manager.cc:1424][EVENT]167075 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:38:47.857.880 [graph_manager.cc:1248][EVENT]167075 PreRun:PreRun start: graph node size 4, session id 23, graph id 22, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:38:47.858.150 [atrace_api.c:28](tid:167075) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:38:47.858.178 [trace_rb_log.c:84](tid:167075) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:38:47.858.192 [atrace_api.c:32](tid:167075) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:38:47.858.208 [client_manager.cpp:157][SetProfilingCallback][tid:167075] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:38:47.858.962 [parallel_partitioner.cc:165][EVENT]167075 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.859.005 [parallel_partitioner.cc:178][EVENT]167075 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.859.050 [graph_prepare.cc:1378][EVENT]167075 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.859.758 [graph_manager.cc:1050][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [725] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.859.789 [graph_manager.cc:1052][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.859.931 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.859.962 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.860.010 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.860.023 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.860.067 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.860.101 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.860.119 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.860.220 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.860.241 [graph_manager.cc:1054][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [439] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.860.453 [graph_manager.cc:1055][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [198] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.861.464 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:38:47.861.494 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.861.506 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.861.515 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [321] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.861.524 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.861.533 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:38:47.861.541 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.861.550 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.861.558 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.863.604 [graph_manager.cc:1056][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3130] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.863.674 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.863.692 [graph_prepare.cc:1982][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [51] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.864.113 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:38:47.864.140 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.152 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.161 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [224] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.170 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.192 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:38:47.864.202 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.210 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.219 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.246 [graph_prepare.cc:1983][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [540] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.864.268 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.864.280 [graph_prepare.cc:1984][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.864.294 [graph_prepare.cc:1985][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.864.309 [graph_prepare.cc:1986][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.864.321 [graph_prepare.cc:1987][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.864.335 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.864.347 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.864.361 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.864.454 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.467 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.476 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.485 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.493 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.502 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.510 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.518 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.527 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.535 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.554 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.563 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.572 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.580 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.589 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.597 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.864.619 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.864.632 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.864.665 [graph_prepare.cc:1988][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [335] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.864.678 [graph_manager.cc:1065][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1038] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.877.398 [graph_manager.cc:1077][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12700] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.877.458 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.877.507 [graph_manager.cc:1080][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [72] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.880.942 [graph_manager.cc:1081][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3419] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.880.984 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.880.999 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.010 [graph_manager.cc:1082][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.041 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.054 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.068 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.099 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.113 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.138 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.151 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.191 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.210 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.240 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.267 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.282 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.294 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.303 [graph_manager.cc:2700][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [268] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.423 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.881.437 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.881.446 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.881.455 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.881.463 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.881.472 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CastRemovePass is [9] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.881.481 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.881.489 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.881.498 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.881.506 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.881.515 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.881.523 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.881.532 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.881.547 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.881.556 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.881.566 [graph_manager.cc:2741][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [243] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.575 [graph_manager.cc:2752][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.597 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.609 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.627 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.642 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.653 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.666 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.685 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.759 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.773 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.784 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.803 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.815 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.834 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.846 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.856 [graph_manager.cc:2810][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [262] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.886 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:47.881.898 [graph_manager.cc:2821][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.881.926 [graph_manager.cc:1087][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [898] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.882.059 [graph_manager.cc:1088][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [121] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.882.109 [graph_manager.cc:1089][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.882.125 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.882.154 [graph_manager.cc:1097][EVENT]167075 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:38:47.882.176 [graph_manager.cc:3325][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.882.553 [engine_place.cc:144][EVENT]167075 Run:The time cost of AIcoreEngine::CheckSupported is [275] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.882.580 [engine_place.cc:144][EVENT]167075 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.882.590 [engine_place.cc:144][EVENT]167075 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.882.678 [graph_manager.cc:3351][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [489] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.882.698 [graph_manager.cc:3364][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.882.771 [engine_partitioner.cc:1139][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.882.790 [engine_partitioner.cc:1142][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.882.940 [engine_partitioner.cc:1148][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [141] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.882.987 [engine_partitioner.cc:1155][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.883.034 [engine_partitioner.cc:1164][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.883.069 [graph_manager.cc:3405][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [358] micro second. [INFO] GE(164043,python):2024-01-10-11:38:47.883.087 [graph_manager.cc:3412][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.054.918 [graph_manager.cc:3422][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [171817] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.054.966 [graph_manager.cc:3428][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.109 [graph_manager.cc:3467][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [123] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.129 [graph_manager.cc:3377][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [172419] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.157 [graph_manager.cc:1106][EVENT]167075 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [172988] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.170 [graph_manager.cc:1115][EVENT]167075 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:38:48.055.193 [graph_manager.cc:1130][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.227 [graph_manager.cc:1131][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.253 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.270 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.280 [graph_manager.cc:2837][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.367 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:48.055.380 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:48.055.389 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:48.055.398 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:48.055.407 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [7] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:48.055.416 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:38:48.055.425 [graph_manager.cc:2864][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [127] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.437 [graph_manager.cc:2872][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.456 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.469 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.485 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.499 [compile_nodes_pass.cc:88][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.508 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.519 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.614 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [77] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.644 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.658 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.672 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.687 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.696 [graph_manager.cc:2927][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [244] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.709 [graph_manager.cc:2937][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.739 [graph_manager.cc:2943][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.754 [graph_manager.cc:2950][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.929 [graph_manager.cc:2958][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.055.961 [graph_manager.cc:1132][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [720] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.056.068 [graph_manager.cc:1135][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [94] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.056.107 [graph_manager.cc:2975][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.056.230 [graph_manager.cc:2981][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [110] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.056.245 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.056.255 [graph_manager.cc:2986][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.056.265 [graph_manager.cc:1136][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [179] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.056.382 [graph_manager.cc:3555][EVENT]167075 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [89] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.056.449 [engine_partitioner.cc:1139][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.056.465 [engine_partitioner.cc:1142][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.056.587 [engine_partitioner.cc:1148][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [112] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.056.631 [engine_partitioner.cc:1155][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.056.673 [engine_partitioner.cc:1164][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.056.697 [graph_builder.cc:865][EVENT]167075 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [286] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.056.780 [graph_builder.cc:288][EVENT]167075 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [65] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.056.888 [graph_builder.cc:293][EVENT]167075 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [94] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.057.084 [model_builder.cc:1133][EVENT]167075 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [103] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.057.365 [block_mem_assigner.cc:4069][EVENT]170066 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164043,python):2024-01-10-11:38:48.057.368 [block_mem_assigner.cc:4069][EVENT]170067 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164043,python):2024-01-10-11:38:48.057.835 [graph_mem_assigner.cc:2166][EVENT]167075 SetInputOffset:[IMAS]AfterAssignMemory : online_22 memoffset[1024], memtype[2], theory_min[2048], zero_copy[1024], total_size[1024], no_reuse[1024], streams[1], topo_mode[DFS], mop[], io_reuse[0:0], alloc_mode[] [INFO] GE(164043,python):2024-01-10-11:38:48.057.938 [model_builder.cc:1144][EVENT]167075 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [832] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.057.966 [model_builder.cc:1152][EVENT]167075 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.057.982 [model_builder.cc:1157][EVENT]167075 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.058.103 [model_builder.cc:1167][EVENT]167075 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [110] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.058.122 [model_builder.cc:1174][EVENT]167075 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.058.143 [model_builder.cc:1180][EVENT]167075 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.058.177 [model_builder.cc:1184][EVENT]167075 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.058.197 [graph_builder.cc:304][EVENT]167075 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [1287] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:38:48.058.447 [logger.cc:1071] 167075 ModelBindStream: model_id=832, stream_id=65, flag=0. [INFO] GE(164043,python):2024-01-10-11:38:48.058.558 [task_generator.cc:804][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.058.630 [task_generator.cc:805][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [55] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.059.491 [task_generator.cc:814][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [847] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.059.518 [task_generator.cc:954][EVENT]167075 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [966] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.059.580 [task_generator.cc:967][EVENT]167075 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [35] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:38:48.059.600 [logger.cc:1084] 167075 ModelUnbindStream: model_id=832, stream_id=65, [INFO] GE(164043,python):2024-01-10-11:38:48.059.656 [graph_builder.cc:310][EVENT]167075 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [1445] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.059.778 [graph_manager.cc:1152][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3491] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.059.797 [graph_manager.cc:1164][EVENT]167075 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:38:48.059.829 [graph_manager.cc:1271][EVENT]167075 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [200961] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.059.840 [graph_manager.cc:1272][EVENT]167075 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:38:48.060.159 [atrace_api.c:93](tid:167075) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:38:48.060.177 [atrace_api.c:95](tid:167075) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:38:48.060.787 [model_introduction.cc:236][EVENT]167075 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(164043,python):2024-01-10-11:38:48.060.810 [model_introduction.cc:294][EVENT]167075 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(164043,python):2024-01-10-11:38:48.060.824 [model_introduction.cc:366][EVENT]167075 Data:model io_info size:222 [INFO] GE(164043,python):2024-01-10-11:38:48.137.793 [graph_var_manager.cc:1424][EVENT]167075 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:38:48.137.891 [graph_manager.cc:1248][EVENT]167075 PreRun:PreRun start: graph node size 5, session id 24, graph id 23, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:38:48.138.165 [atrace_api.c:28](tid:167075) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:38:48.138.195 [trace_rb_log.c:84](tid:167075) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:38:48.138.208 [atrace_api.c:32](tid:167075) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:38:48.138.224 [client_manager.cpp:157][SetProfilingCallback][tid:167075] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:38:48.138.594 [parallel_partitioner.cc:165][EVENT]167075 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.138.631 [parallel_partitioner.cc:178][EVENT]167075 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.138.679 [graph_prepare.cc:1378][EVENT]167075 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.138.882 [graph_manager.cc:1050][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [220] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.138.907 [graph_manager.cc:1052][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.139.074 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.139.130 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.139.178 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.139.192 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.139.236 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.139.250 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.139.268 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.139.371 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.139.391 [graph_manager.cc:1054][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [472] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.139.606 [graph_manager.cc:1055][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [201] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.140.957 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.140.987 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.141.000 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.141.010 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [471] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.141.019 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.141.028 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.141.037 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.141.046 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.141.054 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.145.186 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.145.219 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [0] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.145.232 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.145.242 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [365] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.145.251 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.145.271 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.145.281 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.145.290 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.145.299 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.146.734 [graph_manager.cc:1056][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [7109] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.146.807 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.146.828 [graph_prepare.cc:1982][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [57] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.147.432 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.147.460 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.471 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.481 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [338] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.490 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.499 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.147.507 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.516 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.524 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.576 [graph_prepare.cc:1983][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [735] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.147.602 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.147.614 [graph_prepare.cc:1984][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.147.629 [graph_prepare.cc:1985][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.147.642 [graph_prepare.cc:1986][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.147.653 [graph_prepare.cc:1987][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.147.685 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.147.698 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.147.712 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.147.816 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.827 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.836 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.845 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.853 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.862 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.870 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.879 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.887 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.895 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.903 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.911 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SnapshotPass is [3] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.920 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.928 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [7] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.936 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.944 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of IdentityPass is [5] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.147.967 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.147.981 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.148.016 [graph_prepare.cc:1988][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [336] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.148.030 [graph_manager.cc:1065][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1259] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.160.685 [graph_manager.cc:1077][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12626] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.160.797 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.160.852 [graph_manager.cc:1080][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [127] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.166.499 [graph_manager.cc:1081][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [5629] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.166.542 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.166.558 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.166.570 [graph_manager.cc:1082][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.166.603 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.166.620 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.166.634 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.166.808 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [165] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.166.826 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.166.937 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [99] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.166.954 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.007 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [42] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.028 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.056 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.152 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [86] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.169 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.183 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.193 [graph_manager.cc:2700][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [597] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.474 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of EnterPass is [4] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.167.490 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.167.500 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.167.509 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.167.518 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.167.527 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CastRemovePass is [47] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.167.535 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.167.544 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.167.552 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [10] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.167.560 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.167.569 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [19] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.167.577 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.167.585 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [25] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.167.594 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [11] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.167.602 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.167.612 [graph_manager.cc:2741][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [389] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.621 [graph_manager.cc:2752][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.645 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.658 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.681 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.698 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.710 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.723 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.758 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.773 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.787 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.798 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.813 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.826 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.851 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.865 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.874 [graph_manager.cc:2810][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [234] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.925 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.167.937 [graph_manager.cc:2821][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [54] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.167.964 [graph_manager.cc:1087][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1374] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.168.554 [graph_manager.cc:1088][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [576] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.168.622 [graph_manager.cc:1089][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.168.646 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.168.664 [graph_manager.cc:1097][EVENT]167075 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:38:48.168.686 [graph_manager.cc:3325][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.178.907 [engine_place.cc:144][EVENT]167075 Run:The time cost of AIcoreEngine::CheckSupported is [9966] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.178.940 [engine_place.cc:144][EVENT]167075 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.178.951 [engine_place.cc:144][EVENT]167075 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.179.057 [graph_manager.cc:3351][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10357] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.179.076 [graph_manager.cc:3364][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.179.183 [engine_partitioner.cc:1139][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [31] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.179.216 [engine_partitioner.cc:1142][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.179.417 [engine_partitioner.cc:1148][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [191] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.179.464 [engine_partitioner.cc:1155][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.179.514 [engine_partitioner.cc:1164][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.179.551 [graph_manager.cc:3405][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [438] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.179.569 [graph_manager.cc:3412][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.469.751 [graph_manager.cc:3422][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [290166] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.469.825 [graph_manager.cc:3428][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.064 [graph_manager.cc:3467][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [216] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.085 [graph_manager.cc:3377][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [290973] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.105 [graph_manager.cc:1106][EVENT]167075 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [301426] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.118 [graph_manager.cc:1115][EVENT]167075 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:38:48.470.147 [graph_manager.cc:1130][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.182 [graph_manager.cc:1131][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.215 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.236 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.247 [graph_manager.cc:2837][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [47] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.431 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [39] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.470.445 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [5] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.470.478 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondRemovePass is [6] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.470.487 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of BitcastPass is [4] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.470.496 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [11] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.470.505 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [23] micro second, call num is [11] [INFO] GE(164043,python):2024-01-10-11:38:48.470.516 [graph_manager.cc:2864][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [250] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.529 [graph_manager.cc:2872][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.551 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.567 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.586 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.602 [compile_nodes_pass.cc:88][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.612 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.622 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.747 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [116] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.802 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [43] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.817 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.832 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.848 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.858 [graph_manager.cc:2927][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [313] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.871 [graph_manager.cc:2937][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.886 [graph_manager.cc:2943][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.470.897 [graph_manager.cc:2950][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.471.113 [graph_manager.cc:2958][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [64] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.471.159 [graph_manager.cc:1132][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [963] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.471.263 [graph_manager.cc:1135][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [90] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.471.306 [graph_manager.cc:2975][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.471.337 [graph_manager.cc:2981][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.471.351 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.471.362 [graph_manager.cc:2986][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.471.371 [graph_manager.cc:1136][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [92] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.471.753 [graph_manager.cc:3555][EVENT]167075 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [337] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.471.905 [engine_partitioner.cc:1139][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [40] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.471.942 [engine_partitioner.cc:1142][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.472.135 [engine_partitioner.cc:1148][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [181] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.472.178 [engine_partitioner.cc:1155][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.472.228 [engine_partitioner.cc:1164][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.472.258 [graph_builder.cc:865][EVENT]167075 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [428] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:38:48.472.729 [logger.cc:1071] 167075 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:38:48.472.764 [task_generator.cc:804][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [99] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.472.863 [task_generator.cc:805][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [86] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.474.815 [task_generator.cc:814][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1936] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.474.834 [task_generator.cc:954][EVENT]167075 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2169] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.474.910 [task_generator.cc:967][EVENT]167075 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [41] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:38:48.474.935 [logger.cc:1084] 167075 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:38:48.505.853 [graph_manager.cc:1152][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [34444] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.505.937 [graph_manager.cc:1164][EVENT]167075 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:38:48.505.986 [graph_manager.cc:1271][EVENT]167075 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [367481] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.505.998 [graph_manager.cc:1272][EVENT]167075 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:38:48.506.342 [atrace_api.c:93](tid:167075) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:38:48.506.361 [atrace_api.c:95](tid:167075) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:38:48.721.786 [graph_converter.cc:838][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [67062] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.722.109 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [228] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.724.027 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [1888] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.724.526 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [465] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.724.555 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [496] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.724.844 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [275] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.724.940 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [70] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.725.019 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [58] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.725.597 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [559] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.725.871 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [246] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.725.899 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [274] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.725.974 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [64] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.726.043 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [55] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.726.113 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [56] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.726.392 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [265] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.726.621 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [210] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.726.642 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [232] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.726.715 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [62] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.726.783 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [55] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.726.832 [graph_converter.cc:849][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4959] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.727.734 [graph_converter.cc:853][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [889] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.730.173 [graph_converter.cc:857][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2412] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.730.663 [graph_converter.cc:862][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [456] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.868.389 [graph_var_manager.cc:1424][EVENT]167074 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:38:48.868.492 [graph_manager.cc:1248][EVENT]167074 PreRun:PreRun start: graph node size 5, session id 25, graph id 24, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:38:48.869.112 [atrace_api.c:28](tid:167074) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:38:48.869.168 [trace_rb_log.c:84](tid:167074) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:38:48.869.182 [atrace_api.c:32](tid:167074) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:38:48.869.199 [client_manager.cpp:157][SetProfilingCallback][tid:167074] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:38:48.870.019 [parallel_partitioner.cc:165][EVENT]167074 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.870.066 [parallel_partitioner.cc:178][EVENT]167074 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.870.113 [graph_prepare.cc:1378][EVENT]167074 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.870.426 [graph_manager.cc:1050][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [330] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.870.453 [graph_manager.cc:1052][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.870.621 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.870.652 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.870.699 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.870.712 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.870.755 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.870.769 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.870.788 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.870.939 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.870.961 [graph_manager.cc:1054][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [495] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.871.173 [graph_manager.cc:1055][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [200] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.872.516 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.872.547 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.872.558 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.872.568 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [452] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.872.577 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.872.586 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.872.595 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.872.604 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.872.613 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [8] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.876.990 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.877.022 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.877.034 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.877.044 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [328] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.877.053 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.877.061 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.877.070 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.877.079 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.877.087 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.878.492 [graph_manager.cc:1056][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [7298] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.878.565 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.878.596 [graph_prepare.cc:1982][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [68] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.879.190 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.879.220 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.232 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.241 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [328] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.250 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.259 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.879.267 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.276 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.284 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.336 [graph_prepare.cc:1983][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [726] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.879.362 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.879.374 [graph_prepare.cc:1984][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.879.388 [graph_prepare.cc:1985][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.879.402 [graph_prepare.cc:1986][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.879.413 [graph_prepare.cc:1987][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.879.428 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.879.440 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.879.454 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.879.555 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.568 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.577 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.596 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [3] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.605 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.614 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.622 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.631 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [3] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.639 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.647 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.655 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.664 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.672 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.680 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.688 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.697 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:48.879.719 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.879.732 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.879.766 [graph_prepare.cc:1988][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [342] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.879.778 [graph_manager.cc:1065][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1251] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.892.400 [graph_manager.cc:1077][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12601] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.892.474 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.892.526 [graph_manager.cc:1080][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [88] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.897.917 [graph_manager.cc:1081][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [5376] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.897.961 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.897.976 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.897.999 [graph_manager.cc:1082][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [47] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.898.032 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.898.047 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.898.062 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.898.214 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [141] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.898.230 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.898.327 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [86] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.898.344 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.898.396 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [41] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.898.417 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.898.445 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.898.535 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [79] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.898.553 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.898.566 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.898.576 [graph_manager.cc:2700][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [550] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.898.829 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.898.845 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.898.855 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.898.864 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [4] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.898.873 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.898.882 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CastRemovePass is [42] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.898.890 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [4] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.898.908 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.898.918 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [9] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.898.926 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.898.935 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [19] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.898.943 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [15] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.898.952 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [23] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.898.960 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [10] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.898.969 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.898.979 [graph_manager.cc:2741][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [384] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.898.988 [graph_manager.cc:2752][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.899.012 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.899.025 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.899.047 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.899.063 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.899.076 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.899.089 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.899.110 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.899.125 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.899.138 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.899.148 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.899.160 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.899.172 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.899.201 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.899.216 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.899.225 [graph_manager.cc:2810][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [217] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.899.272 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:48.899.285 [graph_manager.cc:2821][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [52] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.899.314 [graph_manager.cc:1087][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1295] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.899.897 [graph_manager.cc:1088][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [570] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.899.962 [graph_manager.cc:1089][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.899.987 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.900.005 [graph_manager.cc:1097][EVENT]167074 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:38:48.900.028 [graph_manager.cc:3325][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.907.932 [engine_place.cc:144][EVENT]167074 Run:The time cost of AIcoreEngine::CheckSupported is [7680] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.907.965 [engine_place.cc:144][EVENT]167074 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.907.975 [engine_place.cc:144][EVENT]167074 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.908.075 [graph_manager.cc:3351][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [8033] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.908.094 [graph_manager.cc:3364][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.908.175 [engine_partitioner.cc:1139][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.908.208 [engine_partitioner.cc:1142][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.908.415 [engine_partitioner.cc:1148][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [197] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.908.464 [engine_partitioner.cc:1155][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.908.514 [engine_partitioner.cc:1164][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.908.551 [graph_manager.cc:3405][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [444] micro second. [INFO] GE(164043,python):2024-01-10-11:38:48.908.582 [graph_manager.cc:3412][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.023 [graph_manager.cc:3422][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [107425] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.078 [graph_manager.cc:3428][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.255 [graph_manager.cc:3467][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [154] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.276 [graph_manager.cc:3377][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [108169] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.294 [graph_manager.cc:1106][EVENT]167074 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [116274] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.307 [graph_manager.cc:1115][EVENT]167074 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:38:49.016.333 [graph_manager.cc:1130][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.368 [graph_manager.cc:1131][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.397 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.417 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.428 [graph_manager.cc:2837][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [42] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.569 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [31] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:49.016.583 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:49.016.593 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:49.016.602 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:49.016.611 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [7] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:49.016.620 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [13] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:49.016.630 [graph_manager.cc:2864][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [183] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.643 [graph_manager.cc:2872][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.663 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.693 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.711 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.726 [compile_nodes_pass.cc:88][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.736 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.746 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.862 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [106] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.912 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.927 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.942 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.956 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.965 [graph_manager.cc:2927][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [307] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.978 [graph_manager.cc:2937][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.016.993 [graph_manager.cc:2943][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.017.004 [graph_manager.cc:2950][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.017.199 [graph_manager.cc:2958][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [59] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.017.233 [graph_manager.cc:1132][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [850] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.017.316 [graph_manager.cc:1135][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [70] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.017.354 [graph_manager.cc:2975][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.017.390 [graph_manager.cc:2981][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.017.404 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.017.425 [graph_manager.cc:2986][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [24] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.017.435 [graph_manager.cc:1136][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [102] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.017.808 [graph_manager.cc:3555][EVENT]167074 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [330] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.017.950 [engine_partitioner.cc:1139][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.017.983 [engine_partitioner.cc:1142][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.018.163 [engine_partitioner.cc:1148][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [169] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.018.204 [engine_partitioner.cc:1155][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.018.251 [engine_partitioner.cc:1164][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.018.278 [graph_builder.cc:865][EVENT]167074 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [394] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:38:49.018.699 [logger.cc:1071] 167074 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:38:49.018.734 [task_generator.cc:804][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [85] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.018.822 [task_generator.cc:805][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [76] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.020.606 [task_generator.cc:814][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1770] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.020.622 [task_generator.cc:954][EVENT]167074 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1974] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.020.693 [task_generator.cc:967][EVENT]167074 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [39] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:38:49.020.717 [logger.cc:1084] 167074 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:38:49.021.647 [graph_manager.cc:1152][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4176] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.021.682 [graph_manager.cc:1164][EVENT]167074 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:38:49.021.731 [graph_manager.cc:1271][EVENT]167074 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [151801] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.021.743 [graph_manager.cc:1272][EVENT]167074 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:38:49.022.064 [atrace_api.c:93](tid:167074) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:38:49.022.082 [atrace_api.c:95](tid:167074) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:38:49.047.359 [graph_converter.cc:838][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [9977] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.047.596 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [168] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.049.233 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [1608] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.049.746 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [482] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.049.778 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [515] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.050.055 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [264] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.050.144 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [63] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.050.218 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [53] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.050.756 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [522] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.050.993 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [207] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.051.017 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [233] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.051.087 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [58] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.051.150 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [49] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.051.212 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [50] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.051.465 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [240] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.051.673 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [187] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.051.692 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [208] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.051.758 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [55] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.051.818 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [48] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.051.837 [graph_converter.cc:849][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4415] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.052.621 [graph_converter.cc:853][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [772] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.054.768 [graph_converter.cc:857][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2121] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.055.211 [graph_converter.cc:862][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [409] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.139.467 [graph_var_manager.cc:1424][EVENT]167076 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:38:49.139.591 [graph_manager.cc:1248][EVENT]167076 PreRun:PreRun start: graph node size 7, session id 26, graph id 25, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:38:49.140.227 [atrace_api.c:28](tid:167076) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:38:49.140.286 [trace_rb_log.c:84](tid:167076) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:38:49.140.299 [atrace_api.c:32](tid:167076) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:38:49.140.316 [client_manager.cpp:157][SetProfilingCallback][tid:167076] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:38:49.141.026 [parallel_partitioner.cc:165][EVENT]167076 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.141.069 [parallel_partitioner.cc:178][EVENT]167076 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.141.120 [graph_prepare.cc:1378][EVENT]167076 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.141.362 [graph_manager.cc:1050][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [264] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.141.388 [graph_manager.cc:1052][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.141.565 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.141.596 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.141.643 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.141.657 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.141.757 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.141.772 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.141.790 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.141.905 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [4] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.141.926 [graph_manager.cc:1054][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [525] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.142.140 [graph_manager.cc:1055][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [200] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.143.620 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [14] [INFO] GE(164043,python):2024-01-10-11:38:49.143.650 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [6] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.143.662 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of MergePass is [6] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.143.682 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of InferShapePass is [517] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.143.693 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [15] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.143.702 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [4] micro second, call num is [14] [INFO] GE(164043,python):2024-01-10-11:38:49.143.710 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.143.719 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [24] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.143.727 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of InferValuePass is [7] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.145.305 [graph_manager.cc:1056][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3145] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.145.381 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.145.401 [graph_prepare.cc:1982][EVENT]167076 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [61] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.146.098 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [14] [INFO] GE(164043,python):2024-01-10-11:38:49.146.128 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.139 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.148 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of InferShapePass is [440] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.157 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [17] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.166 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [14] [INFO] GE(164043,python):2024-01-10-11:38:49.146.175 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [10] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.183 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.192 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.220 [graph_prepare.cc:1983][EVENT]167076 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [805] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.146.246 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.146.257 [graph_prepare.cc:1984][EVENT]167076 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.146.270 [graph_prepare.cc:1985][EVENT]167076 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.146.286 [graph_prepare.cc:1986][EVENT]167076 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.146.308 [graph_prepare.cc:1987][EVENT]167076 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.146.325 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.146.337 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.146.352 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.146.474 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.487 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.496 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of PrintOpPass is [4] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.505 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.513 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.522 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.530 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.538 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.547 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.555 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.564 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.572 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of SnapshotPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.580 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.588 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.597 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.605 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.146.629 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.146.641 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.146.680 [graph_prepare.cc:1988][EVENT]167076 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [361] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.146.702 [graph_manager.cc:1065][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1362] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.159.577 [graph_manager.cc:1077][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12855] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.159.660 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.159.713 [graph_manager.cc:1080][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [98] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.163.970 [graph_manager.cc:1081][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [4241] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.012 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.027 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.038 [graph_manager.cc:1082][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.070 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.085 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.099 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.134 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [25] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.147 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.164 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.178 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.227 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.247 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.278 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.312 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.327 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.339 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.360 [graph_manager.cc:2700][EVENT]167076 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [296] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.520 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.164.533 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.164.542 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.164.551 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.164.560 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.164.568 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of CastRemovePass is [11] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.164.577 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.164.585 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.164.594 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.164.602 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [5] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.164.610 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.164.618 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.164.627 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.164.635 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.164.643 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.164.653 [graph_manager.cc:2741][EVENT]167076 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [274] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.662 [graph_manager.cc:2752][EVENT]167076 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.684 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.696 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.716 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.732 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.744 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.765 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.785 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.800 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.813 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.824 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.837 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.848 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.869 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.884 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.894 [graph_manager.cc:2810][EVENT]167076 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [213] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.931 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.164.943 [graph_manager.cc:2821][EVENT]167076 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [41] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.164.970 [graph_manager.cc:1087][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [912] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.165.106 [graph_manager.cc:1088][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [123] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.165.152 [graph_manager.cc:1089][EVENT]167076 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.165.170 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.165.214 [graph_manager.cc:1097][EVENT]167076 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:38:49.165.237 [graph_manager.cc:3325][EVENT]167076 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.165.760 [engine_place.cc:144][EVENT]167076 Run:The time cost of AIcoreEngine::CheckSupported is [407] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.165.790 [engine_place.cc:144][EVENT]167076 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.165.800 [engine_place.cc:144][EVENT]167076 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.165.892 [graph_manager.cc:3351][EVENT]167076 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [641] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.165.921 [graph_manager.cc:3364][EVENT]167076 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.165.992 [engine_partitioner.cc:1139][EVENT]167076 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.166.011 [engine_partitioner.cc:1142][EVENT]167076 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.166.265 [engine_partitioner.cc:1148][EVENT]167076 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [243] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.166.318 [engine_partitioner.cc:1155][EVENT]167076 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.166.371 [engine_partitioner.cc:1164][EVENT]167076 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [41] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.166.406 [graph_manager.cc:3405][EVENT]167076 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [472] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.166.423 [graph_manager.cc:3412][EVENT]167076 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.178.431 [graph_manager.cc:3422][EVENT]167076 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [11995] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.178.467 [graph_manager.cc:3428][EVENT]167076 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.178.619 [graph_manager.cc:3467][EVENT]167076 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [131] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.178.638 [graph_manager.cc:3377][EVENT]167076 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [12706] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.178.654 [graph_manager.cc:1106][EVENT]167076 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [13425] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.178.668 [graph_manager.cc:1115][EVENT]167076 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:38:49.178.691 [graph_manager.cc:1130][EVENT]167076 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.178.722 [graph_manager.cc:1131][EVENT]167076 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.178.747 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.178.764 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.178.773 [graph_manager.cc:2837][EVENT]167076 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.178.871 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.178.895 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.178.904 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.178.913 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of BitcastPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.178.922 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [7] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.178.931 [base_pass.cc:339][EVENT]167076 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:38:49.178.941 [graph_manager.cc:2864][EVENT]167076 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [151] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.178.953 [graph_manager.cc:2872][EVENT]167076 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.178.971 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.178.986 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.003 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.017 [compile_nodes_pass.cc:88][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.027 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.037 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.148 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [102] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.178 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.192 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.205 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.218 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.228 [graph_manager.cc:2927][EVENT]167076 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [259] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.240 [graph_manager.cc:2937][EVENT]167076 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.255 [graph_manager.cc:2943][EVENT]167076 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.273 [graph_manager.cc:2950][EVENT]167076 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.450 [graph_manager.cc:2958][EVENT]167076 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [50] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.483 [graph_manager.cc:1132][EVENT]167076 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [747] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.552 [graph_manager.cc:1135][EVENT]167076 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [56] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.591 [graph_manager.cc:2975][EVENT]167076 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.622 [graph_manager.cc:2981][EVENT]167076 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.636 [pass_manager.cc:82][EVENT]167076 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.646 [graph_manager.cc:2986][EVENT]167076 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.655 [graph_manager.cc:1136][EVENT]167076 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [87] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.804 [graph_manager.cc:3555][EVENT]167076 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [112] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.916 [engine_partitioner.cc:1139][EVENT]167076 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.179.933 [engine_partitioner.cc:1142][EVENT]167076 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.180.134 [engine_partitioner.cc:1148][EVENT]167076 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [191] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.180.174 [engine_partitioner.cc:1155][EVENT]167076 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.180.218 [engine_partitioner.cc:1164][EVENT]167076 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.180.242 [graph_builder.cc:865][EVENT]167076 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [376] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:38:49.180.571 [logger.cc:1071] 167076 ModelBindStream: model_id=1344, stream_id=1601, flag=0. [INFO] GE(164043,python):2024-01-10-11:38:49.180.601 [task_generator.cc:804][EVENT]167076 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [78] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.180.675 [task_generator.cc:805][EVENT]167076 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [62] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.181.434 [task_generator.cc:814][EVENT]167076 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [743] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.181.452 [task_generator.cc:954][EVENT]167076 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [929] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.181.518 [task_generator.cc:967][EVENT]167076 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [32] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:38:49.181.537 [logger.cc:1084] 167076 ModelUnbindStream: model_id=1344, stream_id=1601, [INFO] GE(164043,python):2024-01-10-11:38:49.181.758 [graph_manager.cc:1152][EVENT]167076 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2073] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.181.779 [graph_manager.cc:1164][EVENT]167076 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:38:49.181.810 [graph_manager.cc:1271][EVENT]167076 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [40886] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.181.820 [graph_manager.cc:1272][EVENT]167076 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:38:49.182.125 [atrace_api.c:93](tid:167076) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:38:49.182.146 [atrace_api.c:95](tid:167076) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:38:49.189.232 [graph_converter.cc:838][EVENT]167076 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1742] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.189.315 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of ZeroCopy is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.189.978 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of CEM is [642] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.190.256 [copy_flow_launch_fuse.cc:395][EVENT]167076 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [248] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.190.281 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [275] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.190.335 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [42] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.190.369 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.190.403 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.190.617 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of CEM is [202] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.190.723 [copy_flow_launch_fuse.cc:395][EVENT]167076 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [86] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.190.737 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [101] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.190.775 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [28] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.190.806 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.190.838 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.190.950 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of CEM is [102] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.191.041 [copy_flow_launch_fuse.cc:395][EVENT]167076 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [79] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.191.054 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [92] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.191.088 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.191.130 [base_optimizer.cc:70][EVENT]167076 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.191.146 [graph_converter.cc:849][EVENT]167076 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1872] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.191.451 [graph_converter.cc:853][EVENT]167076 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [295] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.192.464 [graph_converter.cc:857][EVENT]167076 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [995] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.192.680 [graph_converter.cc:862][EVENT]167076 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [187] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.257.542 [graph_var_manager.cc:1424][EVENT]167075 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:38:49.257.641 [graph_manager.cc:1248][EVENT]167075 PreRun:PreRun start: graph node size 5, session id 27, graph id 26, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:38:49.257.938 [atrace_api.c:28](tid:167075) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:38:49.257.966 [trace_rb_log.c:84](tid:167075) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:38:49.257.979 [atrace_api.c:32](tid:167075) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:38:49.257.995 [client_manager.cpp:157][SetProfilingCallback][tid:167075] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:38:49.258.355 [parallel_partitioner.cc:165][EVENT]167075 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.258.394 [parallel_partitioner.cc:178][EVENT]167075 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.258.438 [graph_prepare.cc:1378][EVENT]167075 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.258.577 [graph_manager.cc:1050][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [155] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.258.600 [graph_manager.cc:1052][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.258.751 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.258.782 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.258.830 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.258.844 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.258.890 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.258.903 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.258.946 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.259.051 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.259.071 [graph_manager.cc:1054][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [459] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.259.284 [graph_manager.cc:1055][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [200] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.260.473 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:49.260.504 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.260.517 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.260.527 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [392] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.260.536 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.260.545 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:49.260.553 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.260.562 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.260.570 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [7] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.262.905 [graph_manager.cc:1056][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3600] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.262.978 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.262.997 [graph_prepare.cc:1982][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [56] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.263.602 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:49.263.631 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.263.642 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.263.651 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferShapePass is [338] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.263.660 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.263.669 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [4] micro second, call num is [10] [INFO] GE(164043,python):2024-01-10-11:38:49.263.678 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.263.697 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.263.706 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.263.761 [graph_prepare.cc:1983][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [750] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.263.786 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.263.798 [graph_prepare.cc:1984][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.263.812 [graph_prepare.cc:1985][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.263.827 [graph_prepare.cc:1986][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.263.838 [graph_prepare.cc:1987][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.263.851 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.263.863 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.263.877 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.263.979 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.263.991 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.264.001 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.264.009 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.264.018 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.264.026 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.264.035 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [0] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.264.043 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.264.052 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of StopGradientPass is [4] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.264.060 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.264.069 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [3] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.264.086 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.264.095 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [4] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.264.104 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.264.112 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.264.120 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [5] [INFO] GE(164043,python):2024-01-10-11:38:49.264.143 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.264.156 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.264.193 [graph_prepare.cc:1988][EVENT]167075 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [346] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.264.206 [graph_manager.cc:1065][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1266] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.276.354 [graph_manager.cc:1077][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12129] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.276.430 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.276.479 [graph_manager.cc:1080][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [89] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.285.312 [graph_manager.cc:1081][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [8816] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.285.358 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.285.375 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.285.388 [graph_manager.cc:1082][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.285.423 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.285.438 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.285.453 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.285.643 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [178] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.285.662 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.285.829 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [155] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.285.862 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.285.918 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [46] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.285.941 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.285.962 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.069 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [96] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.087 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.101 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.111 [graph_manager.cc:2700][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [694] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.405 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:49.286.422 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:49.286.433 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:49.286.442 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:49.286.451 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:49.286.460 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CastRemovePass is [53] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:49.286.468 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:49.286.477 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:49.286.485 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [11] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:49.286.493 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [4] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:49.286.502 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [21] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:49.286.510 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [15] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:49.286.519 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [24] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:49.286.527 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [12] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:49.286.545 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [6] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:49.286.555 [graph_manager.cc:2741][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [425] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.565 [graph_manager.cc:2752][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.589 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.602 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.627 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.643 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.656 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.670 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.692 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.706 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.719 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.729 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.742 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.753 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.778 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.792 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.801 [graph_manager.cc:2810][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [218] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.855 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:38:49.286.867 [graph_manager.cc:2821][EVENT]167075 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [57] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.286.895 [graph_manager.cc:1087][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1486] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.287.591 [graph_manager.cc:1088][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [682] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.287.660 [graph_manager.cc:1089][EVENT]167075 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.287.697 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.287.717 [graph_manager.cc:1097][EVENT]167075 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:38:49.287.739 [graph_manager.cc:3325][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.298.630 [engine_place.cc:144][EVENT]167075 Run:The time cost of AIcoreEngine::CheckSupported is [10614] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.298.662 [engine_place.cc:144][EVENT]167075 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.298.673 [engine_place.cc:144][EVENT]167075 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.298.776 [graph_manager.cc:3351][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [11022] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.298.796 [graph_manager.cc:3364][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.298.884 [engine_partitioner.cc:1139][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.298.919 [engine_partitioner.cc:1142][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.299.128 [engine_partitioner.cc:1148][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [199] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.299.177 [engine_partitioner.cc:1155][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.299.226 [engine_partitioner.cc:1164][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.299.265 [graph_manager.cc:3405][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [456] micro second. [INFO] GE(164043,python):2024-01-10-11:38:49.299.284 [graph_manager.cc:3412][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. / - \ | / - \ | [INFO] GE(164043,python):2024-01-10-11:40:10.119.754 [graph_manager.cc:3422][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [80820454] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.119.842 [graph_manager.cc:3428][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.120.145 [graph_manager.cc:3467][EVENT]167075 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [273] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.120.167 [graph_manager.cc:3377][EVENT]167075 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [80821359] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.120.193 [graph_manager.cc:1106][EVENT]167075 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [80832461] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.120.227 [graph_manager.cc:1115][EVENT]167075 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:40:10.120.260 [graph_manager.cc:1130][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.120.298 [graph_manager.cc:1131][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [25] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.120.333 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.120.356 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.120.366 [graph_manager.cc:2837][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [50] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.120.564 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [45] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:40:10.120.579 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:40:10.120.589 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of CondRemovePass is [7] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:40:10.120.598 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:40:10.120.607 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [14] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:40:10.120.615 [base_pass.cc:339][EVENT]167075 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [23] micro second, call num is [12] [INFO] GE(164043,python):2024-01-10-11:40:10.120.626 [graph_manager.cc:2864][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [242] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.120.640 [graph_manager.cc:2872][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.120.662 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.120.678 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.120.696 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.120.712 [compile_nodes_pass.cc:88][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.120.723 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.120.733 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.120.869 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [126] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.120.980 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [88] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.120.997 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.121.012 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.121.029 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.121.038 [graph_manager.cc:2927][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [382] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.121.052 [graph_manager.cc:2937][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.121.069 [graph_manager.cc:2943][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.121.081 [graph_manager.cc:2950][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.121.410 [graph_manager.cc:2958][EVENT]167075 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [69] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.121.448 [graph_manager.cc:1132][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [1136] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.121.561 [graph_manager.cc:1135][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [100] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.121.617 [graph_manager.cc:2975][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.121.652 [graph_manager.cc:2981][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.121.667 [pass_manager.cc:82][EVENT]167075 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.121.677 [graph_manager.cc:2986][EVENT]167075 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.121.701 [graph_manager.cc:1136][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [123] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.122.123 [graph_manager.cc:3555][EVENT]167075 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [368] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.122.298 [engine_partitioner.cc:1139][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [42] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.122.341 [engine_partitioner.cc:1142][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.122.548 [engine_partitioner.cc:1148][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [195] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.122.594 [engine_partitioner.cc:1155][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [29] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.122.658 [engine_partitioner.cc:1164][EVENT]167075 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [41] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.122.689 [graph_builder.cc:865][EVENT]167075 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [470] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:40:10.123.266 [logger.cc:1071] 167075 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:40:10.123.305 [task_generator.cc:804][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [155] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.123.417 [task_generator.cc:805][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [99] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.125.567 [task_generator.cc:814][EVENT]167075 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [2134] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.125.583 [task_generator.cc:954][EVENT]167075 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2434] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.125.662 [task_generator.cc:967][EVENT]167075 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [43] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:40:10.125.712 [logger.cc:1084] 167075 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:40:10.127.853 [graph_manager.cc:1152][EVENT]167075 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [6109] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.127.895 [graph_manager.cc:1164][EVENT]167075 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:40:10.127.941 [graph_manager.cc:1271][EVENT]167075 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [80869677] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.127.953 [graph_manager.cc:1272][EVENT]167075 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:40:10.128.382 [atrace_api.c:93](tid:167075) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:40:10.128.407 [atrace_api.c:95](tid:167075) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:40:10.159.043 [graph_converter.cc:838][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [11872] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.159.291 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [191] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.161.361 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [2042] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.161.945 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [551] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.161.974 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [582] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.162.276 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [288] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.162.382 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [79] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.162.468 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [64] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.163.076 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [589] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.163.364 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [245] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.163.390 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [271] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.163.470 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [69] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.163.546 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [59] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.163.620 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of ZeroCopy is [59] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.163.910 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CEM is [276] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.164.154 [copy_flow_launch_fuse.cc:395][EVENT]167075 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [224] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.164.175 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [246] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.164.254 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [67] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.164.328 [base_optimizer.cc:70][EVENT]167075 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [59] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.164.348 [graph_converter.cc:849][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [5256] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.165.259 [graph_converter.cc:853][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [900] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.167.754 [graph_converter.cc:857][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2466] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.168.269 [graph_converter.cc:862][EVENT]167075 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [483] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.255.090 [graph_var_manager.cc:1424][EVENT]167074 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:40:10.255.188 [graph_manager.cc:1248][EVENT]167074 PreRun:PreRun start: graph node size 4, session id 28, graph id 27, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:40:10.255.610 [atrace_api.c:28](tid:167074) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:40:10.255.664 [trace_rb_log.c:84](tid:167074) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:40:10.255.678 [atrace_api.c:32](tid:167074) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:40:10.255.698 [client_manager.cpp:157][SetProfilingCallback][tid:167074] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:40:10.256.231 [parallel_partitioner.cc:165][EVENT]167074 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.256.269 [parallel_partitioner.cc:178][EVENT]167074 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.256.316 [graph_prepare.cc:1378][EVENT]167074 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.256.513 [graph_manager.cc:1050][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [215] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.256.538 [graph_manager.cc:1052][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.256.678 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.256.708 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.256.757 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.256.770 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.256.815 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.256.828 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.256.844 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.256.955 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.256.975 [graph_manager.cc:1054][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [424] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.257.187 [graph_manager.cc:1055][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [199] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.258.268 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:40:10.258.300 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.258.311 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.258.321 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [332] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.258.331 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.258.339 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:40:10.258.348 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.258.357 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.258.365 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [9] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.260.496 [graph_manager.cc:1056][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3288] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.260.566 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.260.597 [graph_prepare.cc:1982][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [64] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.261.114 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:40:10.261.142 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.153 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.162 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [276] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.171 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.180 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164043,python):2024-01-10-11:40:10.261.188 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.197 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.205 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.250 [graph_prepare.cc:1983][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [639] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.261.274 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.261.286 [graph_prepare.cc:1984][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.261.300 [graph_prepare.cc:1985][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.261.313 [graph_prepare.cc:1986][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.261.324 [graph_prepare.cc:1987][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.261.339 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.261.351 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.261.366 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.261.459 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.471 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.479 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.499 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.508 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.516 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.525 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.533 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.542 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.550 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.559 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.567 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.576 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.584 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.592 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.600 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.261.623 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.261.635 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.261.667 [graph_prepare.cc:1988][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [333] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.261.679 [graph_manager.cc:1065][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1147] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.275.332 [graph_manager.cc:1077][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13619] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.275.434 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.275.485 [graph_manager.cc:1080][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [115] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.284.708 [graph_manager.cc:1081][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [9208] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.284.752 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.284.782 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.284.794 [graph_manager.cc:1082][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [51] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.284.827 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.284.843 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.284.858 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.284.960 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [92] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.284.976 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.023 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.038 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.078 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [29] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.097 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.123 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.154 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.169 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.182 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.192 [graph_manager.cc:2700][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [370] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.320 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.285.334 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.285.343 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.285.352 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.285.361 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.285.370 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.285.378 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.285.396 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.285.405 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.285.414 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.285.422 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.285.430 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.285.438 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.285.447 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.285.455 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.285.465 [graph_manager.cc:2741][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [254] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.475 [graph_manager.cc:2752][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.499 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.511 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.529 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.544 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.555 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.568 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.587 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.601 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.615 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.625 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.638 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.649 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.674 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.697 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.709 [graph_manager.cc:2810][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [215] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.741 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.285.753 [graph_manager.cc:2821][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.782 [graph_manager.cc:1087][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [968] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.950 [graph_manager.cc:1088][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [152] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.285.992 [graph_manager.cc:1089][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.286.009 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.286.025 [graph_manager.cc:1097][EVENT]167074 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:40:10.286.046 [graph_manager.cc:3325][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.286.459 [engine_place.cc:144][EVENT]167074 Run:The time cost of AIcoreEngine::CheckSupported is [291] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.286.487 [engine_place.cc:144][EVENT]167074 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.286.497 [engine_place.cc:144][EVENT]167074 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.286.579 [graph_manager.cc:3351][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [519] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.286.598 [graph_manager.cc:3364][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.286.672 [engine_partitioner.cc:1139][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.286.690 [engine_partitioner.cc:1142][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.286.846 [engine_partitioner.cc:1148][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [146] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.286.893 [engine_partitioner.cc:1155][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.286.939 [engine_partitioner.cc:1164][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.286.988 [graph_manager.cc:3405][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [377] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.287.006 [graph_manager.cc:3412][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.304.822 [graph_manager.cc:3422][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [17802] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.304.880 [graph_manager.cc:3428][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.037 [graph_manager.cc:3467][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [134] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.058 [graph_manager.cc:3377][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [18448] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.075 [graph_manager.cc:1106][EVENT]167074 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [19036] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.088 [graph_manager.cc:1115][EVENT]167074 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:40:10.305.113 [graph_manager.cc:1130][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.146 [graph_manager.cc:1131][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.172 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.189 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.198 [graph_manager.cc:2837][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.290 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.305.304 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.305.313 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.305.322 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of BitcastPass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.305.332 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.305.341 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [10] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:40:10.305.351 [graph_manager.cc:2864][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [135] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.363 [graph_manager.cc:2872][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.396 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.411 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.427 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.442 [compile_nodes_pass.cc:88][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.452 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.462 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.545 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [73] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.575 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.588 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.602 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.615 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.624 [graph_manager.cc:2927][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [232] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.637 [graph_manager.cc:2937][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.651 [graph_manager.cc:2943][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.662 [graph_manager.cc:2950][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.883 [graph_manager.cc:2958][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [42] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.916 [graph_manager.cc:1132][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [756] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.305.991 [graph_manager.cc:1135][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [62] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.306.025 [graph_manager.cc:2975][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.306.054 [graph_manager.cc:2981][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.306.069 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.306.089 [graph_manager.cc:2986][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.306.098 [graph_manager.cc:1136][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [91] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.306.239 [graph_manager.cc:3555][EVENT]167074 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [105] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.306.337 [engine_partitioner.cc:1139][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.306.352 [engine_partitioner.cc:1142][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.306.482 [engine_partitioner.cc:1148][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [120] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.306.516 [engine_partitioner.cc:1155][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.306.557 [engine_partitioner.cc:1164][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.306.580 [graph_builder.cc:865][EVENT]167074 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [282] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:40:10.306.899 [logger.cc:1071] 167074 ModelBindStream: model_id=1856, stream_id=65, flag=0. [INFO] GE(164043,python):2024-01-10-11:40:10.306.932 [task_generator.cc:804][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [83] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.306.995 [task_generator.cc:805][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [51] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.307.682 [task_generator.cc:814][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [672] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.307.696 [task_generator.cc:954][EVENT]167074 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [849] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.307.753 [task_generator.cc:967][EVENT]167074 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [32] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:40:10.307.772 [logger.cc:1084] 167074 ModelUnbindStream: model_id=1856, stream_id=65, [INFO] GE(164043,python):2024-01-10-11:40:10.308.365 [graph_manager.cc:1152][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2238] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.308.398 [graph_manager.cc:1164][EVENT]167074 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:40:10.308.436 [graph_manager.cc:1271][EVENT]167074 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [52298] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.308.448 [graph_manager.cc:1272][EVENT]167074 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:40:10.308.764 [atrace_api.c:93](tid:167074) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:40:10.308.781 [atrace_api.c:95](tid:167074) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:40:10.320.665 [graph_converter.cc:838][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [3816] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.320.845 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [120] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.321.347 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [478] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.321.564 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [190] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.321.586 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [215] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.321.870 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [271] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.321.913 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.321.945 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.322.146 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [189] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.322.233 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [68] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.322.249 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [84] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.322.279 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.322.304 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.322.331 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.322.410 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [68] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.322.480 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [58] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.322.491 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [69] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.322.517 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.322.541 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.322.555 [graph_converter.cc:849][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1836] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.322.785 [graph_converter.cc:853][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [221] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.323.530 [graph_converter.cc:857][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [729] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.323.683 [graph_converter.cc:862][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [127] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.394.670 [graph_var_manager.cc:1424][EVENT]167073 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:40:10.394.793 [graph_manager.cc:1248][EVENT]167073 PreRun:PreRun start: graph node size 7, session id 29, graph id 28, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:40:10.395.403 [atrace_api.c:28](tid:167073) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:40:10.395.460 [trace_rb_log.c:84](tid:167073) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:40:10.395.473 [atrace_api.c:32](tid:167073) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:40:10.395.490 [client_manager.cpp:157][SetProfilingCallback][tid:167073] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:40:10.395.858 [parallel_partitioner.cc:165][EVENT]167073 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.395.899 [parallel_partitioner.cc:178][EVENT]167073 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.395.948 [graph_prepare.cc:1378][EVENT]167073 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.396.100 [graph_manager.cc:1050][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [170] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.396.124 [graph_manager.cc:1052][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.396.296 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.396.327 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.396.375 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.396.388 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.396.436 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.396.449 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.396.467 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.396.574 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [5] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.396.594 [graph_manager.cc:1054][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [459] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.396.800 [graph_manager.cc:1055][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [193] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.398.284 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [14] [INFO] GE(164043,python):2024-01-10-11:40:10.398.316 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.398.328 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.398.349 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferShapePass is [552] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.398.359 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [18] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.398.367 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [14] [INFO] GE(164043,python):2024-01-10-11:40:10.398.376 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.398.384 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [21] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.398.393 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferValuePass is [7] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.399.983 [graph_manager.cc:1056][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3162] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.400.059 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.400.079 [graph_prepare.cc:1982][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [62] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.400.749 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [14] [INFO] GE(164043,python):2024-01-10-11:40:10.400.776 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.400.788 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.400.797 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferShapePass is [418] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.400.806 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [15] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.400.815 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [14] [INFO] GE(164043,python):2024-01-10-11:40:10.400.824 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [9] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.400.832 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.400.841 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.400.869 [graph_prepare.cc:1983][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [775] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.400.893 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.400.905 [graph_prepare.cc:1984][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.400.919 [graph_prepare.cc:1985][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.400.934 [graph_prepare.cc:1986][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.400.955 [graph_prepare.cc:1987][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.400.971 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.400.983 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.400.998 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.401.121 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.401.133 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondPass is [6] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.401.142 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.401.151 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.401.159 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.401.168 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.401.176 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.401.184 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.401.193 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.401.201 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.401.209 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.401.217 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.401.226 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.401.234 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [8] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.401.242 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.401.251 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.401.276 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.401.289 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.401.326 [graph_prepare.cc:1988][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [360] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.401.346 [graph_manager.cc:1065][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1330] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.414.236 [graph_manager.cc:1077][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12870] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.414.318 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.414.369 [graph_manager.cc:1080][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [95] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.418.608 [graph_manager.cc:1081][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [4221] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.418.651 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.418.667 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.418.678 [graph_manager.cc:1082][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.418.710 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.418.725 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.418.739 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.418.775 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.418.789 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.418.806 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.418.820 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.418.868 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.418.889 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.418.919 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.418.953 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.418.968 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.418.980 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.012 [graph_manager.cc:2700][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [306] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.171 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.419.185 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.419.195 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.419.205 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.419.213 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.419.222 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CastRemovePass is [12] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.419.230 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [4] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.419.239 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.419.248 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [4] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.419.256 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.419.265 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.419.273 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.419.281 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.419.290 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.419.298 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [5] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.419.307 [graph_manager.cc:2741][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [277] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.316 [graph_manager.cc:2752][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.339 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.351 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.371 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.386 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.398 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.419 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.439 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.454 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.468 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.479 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.492 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.503 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.525 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.539 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.548 [graph_manager.cc:2810][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [213] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.586 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.419.598 [graph_manager.cc:2821][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [42] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.625 [graph_manager.cc:1087][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [927] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.760 [graph_manager.cc:1088][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [122] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.806 [graph_manager.cc:1089][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.823 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.419.865 [graph_manager.cc:1097][EVENT]167073 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:40:10.419.888 [graph_manager.cc:3325][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.420.400 [engine_place.cc:144][EVENT]167073 Run:The time cost of AIcoreEngine::CheckSupported is [394] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.420.429 [engine_place.cc:144][EVENT]167073 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.420.439 [engine_place.cc:144][EVENT]167073 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.420.529 [graph_manager.cc:3351][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [627] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.420.559 [graph_manager.cc:3364][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.420.629 [engine_partitioner.cc:1139][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [24] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.420.647 [engine_partitioner.cc:1142][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.420.890 [engine_partitioner.cc:1148][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [233] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.420.941 [engine_partitioner.cc:1155][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.420.993 [engine_partitioner.cc:1164][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [40] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.421.031 [graph_manager.cc:3405][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [459] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.421.050 [graph_manager.cc:3412][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.432.763 [graph_manager.cc:3422][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [11699] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.432.804 [graph_manager.cc:3428][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.432.956 [graph_manager.cc:3467][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [132] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.432.976 [graph_manager.cc:3377][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [12405] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.432.992 [graph_manager.cc:1106][EVENT]167073 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [13111] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.006 [graph_manager.cc:1115][EVENT]167073 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:40:10.433.030 [graph_manager.cc:1130][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.061 [graph_manager.cc:1131][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.088 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.104 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.114 [graph_manager.cc:2837][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [36] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.216 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.433.240 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.433.250 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.433.259 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.433.268 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [9] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.433.276 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [7] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.433.286 [graph_manager.cc:2864][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [154] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.298 [graph_manager.cc:2872][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.318 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.333 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.348 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.362 [compile_nodes_pass.cc:88][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.372 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.382 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.494 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [103] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.525 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.539 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.553 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.567 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.576 [graph_manager.cc:2927][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [260] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.589 [graph_manager.cc:2937][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.603 [graph_manager.cc:2943][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.623 [graph_manager.cc:2950][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.838 [graph_manager.cc:2958][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [52] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.873 [graph_manager.cc:1132][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [798] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.944 [graph_manager.cc:1135][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [58] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.433.985 [graph_manager.cc:2975][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.434.020 [graph_manager.cc:2981][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.434.034 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.434.044 [graph_manager.cc:2986][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.434.053 [graph_manager.cc:1136][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [93] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.434.196 [graph_manager.cc:3555][EVENT]167073 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [105] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.434.303 [engine_partitioner.cc:1139][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.434.321 [engine_partitioner.cc:1142][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.434.513 [engine_partitioner.cc:1148][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [182] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.434.554 [engine_partitioner.cc:1155][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.434.599 [engine_partitioner.cc:1164][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.434.623 [graph_builder.cc:865][EVENT]167073 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [366] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:40:10.434.951 [logger.cc:1071] 167073 ModelBindStream: model_id=64, stream_id=321, flag=0. [INFO] GE(164043,python):2024-01-10-11:40:10.434.983 [task_generator.cc:804][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [78] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.435.057 [task_generator.cc:805][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [62] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.435.782 [task_generator.cc:814][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [710] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.435.797 [task_generator.cc:954][EVENT]167073 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [893] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.435.863 [task_generator.cc:967][EVENT]167073 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [32] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:40:10.435.883 [logger.cc:1084] 167073 ModelUnbindStream: model_id=64, stream_id=321, [INFO] GE(164043,python):2024-01-10-11:40:10.436.058 [graph_manager.cc:1152][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [1974] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.436.076 [graph_manager.cc:1164][EVENT]167073 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:40:10.436.107 [graph_manager.cc:1271][EVENT]167073 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [40346] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.436.118 [graph_manager.cc:1272][EVENT]167073 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:40:10.436.429 [atrace_api.c:93](tid:167073) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:40:10.436.446 [atrace_api.c:95](tid:167073) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:40:10.443.676 [graph_converter.cc:838][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1744] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.443.761 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [38] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.444.389 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [608] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.444.669 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [254] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.444.693 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [279] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.444.748 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [44] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.444.782 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.444.816 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.445.025 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [198] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.445.131 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [86] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.445.147 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [102] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.445.184 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [27] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.445.214 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.445.247 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.445.359 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [102] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.445.450 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [79] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.445.462 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [92] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.445.496 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [25] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.445.538 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.445.552 [graph_converter.cc:849][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1835] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.445.910 [graph_converter.cc:853][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [348] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.446.936 [graph_converter.cc:857][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1005] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.447.153 [graph_converter.cc:862][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [186] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.522.705 [graph_var_manager.cc:1424][EVENT]167074 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:40:10.522.805 [graph_manager.cc:1248][EVENT]167074 PreRun:PreRun start: graph node size 7, session id 30, graph id 29, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:40:10.523.075 [atrace_api.c:28](tid:167074) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:40:10.523.104 [trace_rb_log.c:84](tid:167074) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:40:10.523.117 [atrace_api.c:32](tid:167074) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:40:10.523.133 [client_manager.cpp:157][SetProfilingCallback][tid:167074] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:40:10.523.484 [parallel_partitioner.cc:165][EVENT]167074 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.523.524 [parallel_partitioner.cc:178][EVENT]167074 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.523.572 [graph_prepare.cc:1378][EVENT]167074 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.523.706 [graph_manager.cc:1050][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [152] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.523.730 [graph_manager.cc:1052][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.523.903 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [4] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.523.935 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.523.981 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.523.994 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.524.038 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.524.052 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.524.095 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.524.204 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [5] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.524.225 [graph_manager.cc:1054][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [482] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.524.433 [graph_manager.cc:1055][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [196] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.525.902 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [14] [INFO] GE(164043,python):2024-01-10-11:40:10.525.934 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.525.946 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.525.956 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [550] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.525.965 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [17] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.525.974 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [4] micro second, call num is [14] [INFO] GE(164043,python):2024-01-10-11:40:10.525.983 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.525.991 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [24] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.526.000 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [8] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.527.603 [graph_manager.cc:1056][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3150] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.527.679 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondRemovePass is [7] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.527.699 [graph_prepare.cc:1982][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [61] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.528.418 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [14] [INFO] GE(164043,python):2024-01-10-11:40:10.528.446 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.457 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.467 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [429] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.476 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.485 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [14] [INFO] GE(164043,python):2024-01-10-11:40:10.528.493 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.513 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.523 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.580 [graph_prepare.cc:1983][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [868] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.528.606 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.528.617 [graph_prepare.cc:1984][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.528.631 [graph_prepare.cc:1985][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.528.646 [graph_prepare.cc:1986][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.528.659 [graph_prepare.cc:1987][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.528.674 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.528.686 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.528.700 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.528.821 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.832 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondPass is [5] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.841 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.850 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.858 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.867 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.875 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.883 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.892 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of StopGradientPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.900 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.908 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.925 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.934 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.943 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [7] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.951 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.959 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.528.983 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.528.997 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.529.035 [graph_prepare.cc:1988][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [368] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.529.048 [graph_manager.cc:1065][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1410] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.541.969 [graph_manager.cc:1077][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12900] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.542.101 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.542.157 [graph_manager.cc:1080][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [150] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.546.311 [graph_manager.cc:1081][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [4136] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.546.354 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.546.369 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.546.381 [graph_manager.cc:1082][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.546.414 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.546.429 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.546.443 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.546.476 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.546.490 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.546.507 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.546.533 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.546.584 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [40] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.546.604 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.546.635 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.546.668 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.546.684 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.546.697 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.546.706 [graph_manager.cc:2700][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [298] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.546.864 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of EnterPass is [4] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.546.878 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.546.887 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.546.896 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.546.905 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.546.913 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CastRemovePass is [9] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.546.921 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.546.930 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.546.938 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.546.946 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.546.955 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.546.963 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.546.971 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.546.979 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.546.988 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.547.004 [graph_manager.cc:2741][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [280] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.014 [graph_manager.cc:2752][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.038 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.050 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.070 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.085 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.097 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.109 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.130 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.144 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.157 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.168 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.180 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.192 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.213 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.227 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.237 [graph_manager.cc:2810][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [204] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.276 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.547.287 [graph_manager.cc:2821][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [41] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.314 [graph_manager.cc:1087][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [913] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.450 [graph_manager.cc:1088][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [124] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.497 [graph_manager.cc:1089][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.524 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.547.566 [graph_manager.cc:1097][EVENT]167074 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:40:10.547.589 [graph_manager.cc:3325][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.548.075 [engine_place.cc:144][EVENT]167074 Run:The time cost of AIcoreEngine::CheckSupported is [374] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.548.103 [engine_place.cc:144][EVENT]167074 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.548.112 [engine_place.cc:144][EVENT]167074 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.548.202 [graph_manager.cc:3351][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [599] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.548.221 [graph_manager.cc:3364][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.548.291 [engine_partitioner.cc:1139][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [24] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.548.310 [engine_partitioner.cc:1142][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.548.553 [engine_partitioner.cc:1148][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [233] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.548.604 [engine_partitioner.cc:1155][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [35] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.548.656 [engine_partitioner.cc:1164][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [40] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.548.692 [graph_manager.cc:3405][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [458] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.548.709 [graph_manager.cc:3412][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.560.745 [graph_manager.cc:3422][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [12022] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.560.796 [graph_manager.cc:3428][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.560.972 [graph_manager.cc:3467][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [154] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.560.993 [graph_manager.cc:3377][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [12761] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.011 [graph_manager.cc:1106][EVENT]167074 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [13430] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.036 [graph_manager.cc:1115][EVENT]167074 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:40:10.561.062 [graph_manager.cc:1130][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.096 [graph_manager.cc:1131][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.124 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.142 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.152 [graph_manager.cc:2837][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [39] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.256 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [22] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.561.270 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.561.279 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.561.288 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.561.297 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.561.305 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [7] [INFO] GE(164043,python):2024-01-10-11:40:10.561.315 [graph_manager.cc:2864][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [146] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.327 [graph_manager.cc:2872][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.347 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.361 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.378 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.392 [compile_nodes_pass.cc:88][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.401 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.411 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.526 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [106] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.560 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.583 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.598 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.612 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.621 [graph_manager.cc:2927][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [278] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.634 [graph_manager.cc:2937][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.649 [graph_manager.cc:2943][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.660 [graph_manager.cc:2950][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.861 [graph_manager.cc:2958][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [53] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.897 [graph_manager.cc:1132][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [786] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.561.975 [graph_manager.cc:1135][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [65] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.562.016 [graph_manager.cc:2975][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.562.047 [graph_manager.cc:2981][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.562.062 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.562.072 [graph_manager.cc:2986][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.562.081 [graph_manager.cc:1136][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [90] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.562.242 [graph_manager.cc:3555][EVENT]167074 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [122] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.562.355 [engine_partitioner.cc:1139][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [24] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.562.373 [engine_partitioner.cc:1142][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.562.569 [engine_partitioner.cc:1148][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [186] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.562.610 [engine_partitioner.cc:1155][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [27] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.562.666 [engine_partitioner.cc:1164][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.562.690 [graph_builder.cc:865][EVENT]167074 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [382] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:40:10.563.023 [logger.cc:1071] 167074 ModelBindStream: model_id=1344, stream_id=1601, flag=0. [INFO] GE(164043,python):2024-01-10-11:40:10.563.056 [task_generator.cc:804][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [71] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.563.136 [task_generator.cc:805][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [67] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.563.922 [task_generator.cc:814][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [770] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.563.937 [task_generator.cc:954][EVENT]167074 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [952] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.563.995 [task_generator.cc:967][EVENT]167074 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [33] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:40:10.564.014 [logger.cc:1084] 167074 ModelUnbindStream: model_id=1344, stream_id=1601, [INFO] GE(164043,python):2024-01-10-11:40:10.564.192 [graph_manager.cc:1152][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2079] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.564.211 [graph_manager.cc:1164][EVENT]167074 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:40:10.564.245 [graph_manager.cc:1271][EVENT]167074 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [40859] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.564.256 [graph_manager.cc:1272][EVENT]167074 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:40:10.564.565 [atrace_api.c:93](tid:167074) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:40:10.564.582 [atrace_api.c:95](tid:167074) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:40:10.571.996 [graph_converter.cc:838][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1732] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.572.080 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [37] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.572.718 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [618] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.572.994 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [250] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.573.018 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [274] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.573.073 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [43] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.573.107 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.573.141 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.573.356 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [204] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.573.463 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [87] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.573.488 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [112] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.573.528 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [29] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.573.558 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.573.591 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [23] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.573.737 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [135] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.573.834 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [83] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.573.847 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [96] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.573.883 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.573.914 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.573.929 [graph_converter.cc:849][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1891] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.574.239 [graph_converter.cc:853][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [301] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.575.273 [graph_converter.cc:857][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1015] micro second. [INFO] GE(164043,python):2024-01-10-11:40:10.575.491 [graph_converter.cc:862][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [189] micro second. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:10.579.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 1, total running count: 1 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:27.365.800 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:27.368.454 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 102 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:27.387.044 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 2, total running count: 2 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:27.779.759 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:27.781.847 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 103 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:27.795.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 3, total running count: 3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:27.795.859 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:27.797.585 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 104 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:27.811.252 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 4, total running count: 4 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:27.811.611 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:27.814.388 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 105 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:27.826.736 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 5, total running count: 5 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:27.827.099 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:27.828.866 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 106 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:27.842.177 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 6, total running count: 6 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:27.842.533 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:27.844.917 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 107 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:27.857.350 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 7, total running count: 7 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:27.857.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:27.859.543 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 108 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:27.872.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 8, total running count: 8 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:27.873.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:27.875.103 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 109 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:27.888.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 9, total running count: 9 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:27.888.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:27.890.898 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 110 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:27.903.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 10, total running count: 10 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:27.903.727 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:27.905.657 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 111 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:27.918.378 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 11, total running count: 11 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:27.918.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:27.921.363 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 112 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:27.933.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 12, total running count: 12 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:27.933.567 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:27.935.899 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 113 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:27.948.112 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 13, total running count: 13 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:27.948.473 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:27.950.614 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 114 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:27.962.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 14, total running count: 14 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:27.963.165 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:27.965.414 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 115 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:27.977.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 15, total running count: 15 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:27.977.978 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:27.980.144 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 116 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:27.992.458 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 16, total running count: 16 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:27.992.796 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:27.994.747 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 117 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.007.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 17, total running count: 17 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.007.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.009.462 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 118 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:28.022.122 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 18, total running count: 18 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.022.657 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.025.400 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 119 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.037.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 19, total running count: 19 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.037.567 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.040.060 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 120 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:28.052.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 20, total running count: 20 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.052.531 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.054.696 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 121 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.067.078 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 21, total running count: 21 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.067.433 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.069.455 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 122 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.081.987 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 22, total running count: 22 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.082.640 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.084.359 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 123 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:28.097.109 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 23, total running count: 23 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.097.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.100.295 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 124 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.111.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 24, total running count: 24 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.112.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.114.889 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 125 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.126.718 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 25, total running count: 25 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.127.210 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.129.325 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 126 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.141.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 26, total running count: 26 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.142.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.144.754 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 127 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.156.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 27, total running count: 27 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:28.157.033 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.158.991 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 128 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:28.171.530 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 28, total running count: 28 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.171.939 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.174.681 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 129 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.186.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 29, total running count: 29 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:28.186.827 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.189.091 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 130 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.201.443 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 30, total running count: 30 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.201.890 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.204.053 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 131 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.216.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 31, total running count: 31 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.216.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.218.729 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 132 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.231.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 32, total running count: 32 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.231.574 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.234.251 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 133 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.246.191 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 33, total running count: 33 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.246.693 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.248.610 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 134 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:28.261.106 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 34, total running count: 34 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.261.665 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.264.148 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 135 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.276.156 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 35, total running count: 35 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:28.276.550 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.278.418 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 136 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.291.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 36, total running count: 36 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.291.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.293.799 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 137 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.305.813 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 37, total running count: 37 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:28.306.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.308.178 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 138 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.320.874 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 38, total running count: 38 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.321.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.323.830 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 139 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.335.736 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 39, total running count: 39 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.336.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.338.395 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 140 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.350.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 40, total running count: 40 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.351.244 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.352.824 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 141 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.365.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 41, total running count: 41 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:28.366.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.368.697 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 142 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.380.613 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 42, total running count: 42 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.381.352 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.383.065 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 143 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.396.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 43, total running count: 43 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.396.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.398.711 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 144 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.411.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 44, total running count: 44 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.411.420 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.413.634 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 145 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.426.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 45, total running count: 45 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.426.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.428.139 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 146 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.440.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 46, total running count: 46 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.441.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.443.824 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 147 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.455.740 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 47, total running count: 47 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.456.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.458.357 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 148 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.470.691 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 48, total running count: 48 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.471.413 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.473.073 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 149 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:28.485.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 49, total running count: 49 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.486.402 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.488.713 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 150 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.500.721 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 50, total running count: 50 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.501.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.503.090 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 151 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.515.933 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 51, total running count: 51 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:28.516.493 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.518.863 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 152 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:28.530.897 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 52, total running count: 52 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.531.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.533.598 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 153 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.546.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 53, total running count: 53 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.546.789 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.549.332 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 154 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.561.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 54, total running count: 54 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.561.785 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.563.912 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 155 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.576.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 55, total running count: 55 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:28.576.749 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.578.654 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 156 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.591.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 56, total running count: 56 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:28.591.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.594.299 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 157 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.606.404 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 57, total running count: 57 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.606.769 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.608.671 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 158 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.621.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 58, total running count: 58 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.621.783 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.623.999 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 159 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:28.636.107 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 59, total running count: 59 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.636.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.638.383 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 160 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.651.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 60, total running count: 60 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.651.680 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.653.929 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 161 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.666.071 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 61, total running count: 61 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.666.616 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.668.617 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 162 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.681.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 62, total running count: 62 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.681.592 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.683.253 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 163 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.696.141 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 63, total running count: 63 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:28.696.588 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.698.601 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 164 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.710.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 64, total running count: 64 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.711.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.713.120 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 165 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.725.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 65, total running count: 65 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.726.624 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.728.558 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 166 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.741.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 66, total running count: 66 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.741.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.743.209 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 167 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.755.966 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 67, total running count: 67 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.756.479 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.759.074 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 168 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.771.053 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 68, total running count: 68 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.771.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.773.499 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 169 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:28.785.843 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 69, total running count: 69 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.786.290 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.787.986 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 170 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.800.770 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 70, total running count: 70 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.801.142 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.803.493 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 171 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.815.592 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 71, total running count: 71 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.816.163 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.817.961 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 172 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.830.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 72, total running count: 72 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.830.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.832.624 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 173 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.845.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 73, total running count: 73 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.845.878 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.848.231 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 174 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:28.860.173 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 74, total running count: 74 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.860.848 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.862.669 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 175 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.875.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 75, total running count: 75 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.875.833 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.878.510 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 176 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.890.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 76, total running count: 76 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.890.942 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.893.176 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 177 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.905.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 77, total running count: 77 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.905.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.907.583 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 178 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.920.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 78, total running count: 78 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.920.777 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.923.079 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 179 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.935.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 79, total running count: 79 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.935.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.937.414 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 180 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:28.950.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 80, total running count: 80 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:28.950.709 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.953.153 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 181 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:28.965.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 81, total running count: 81 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.965.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.967.807 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 182 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.980.263 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 82, total running count: 82 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.980.689 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.982.256 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 183 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:28.995.231 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 83, total running count: 83 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:28.995.740 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:28.997.970 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 184 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.010.389 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 84, total running count: 84 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.010.738 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.012.277 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 185 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.025.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 85, total running count: 85 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.025.645 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.027.936 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 186 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.040.073 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 86, total running count: 86 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.040.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.042.772 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 187 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.055.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 87, total running count: 87 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.055.465 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.057.038 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 188 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.070.105 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 88, total running count: 88 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.070.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.072.596 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 189 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.085.071 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 89, total running count: 89 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.085.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.086.974 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 190 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.099.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 90, total running count: 90 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.100.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.102.787 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 191 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.114.749 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 91, total running count: 91 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.115.353 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.117.364 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 192 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.129.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 92, total running count: 92 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.130.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.133.007 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 193 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.144.998 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 93, total running count: 93 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.145.389 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.147.571 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 194 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.159.719 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 94, total running count: 94 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.160.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.162.147 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 195 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.174.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 95, total running count: 95 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.175.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.177.010 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 196 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.189.731 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 96, total running count: 96 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.190.225 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.192.573 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 197 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.204.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 97, total running count: 97 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.205.078 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.206.790 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 198 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.219.687 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 98, total running count: 98 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.220.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.222.290 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 199 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.234.539 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 99, total running count: 99 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.234.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.236.726 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 200 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.249.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 100, total running count: 100 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.249.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.252.355 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 201 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.264.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 101, total running count: 101 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.264.777 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.267.107 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 202 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.279.263 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 102, total running count: 102 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.279.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.281.715 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 203 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.294.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 103, total running count: 103 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.294.715 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.297.406 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 204 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.309.150 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 104, total running count: 104 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.309.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.311.888 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 205 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.324.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 105, total running count: 105 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.324.666 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.326.648 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 206 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.339.153 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 106, total running count: 106 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.339.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.342.359 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 207 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.354.121 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 107, total running count: 107 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.354.630 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.356.721 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 208 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.369.177 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 108, total running count: 108 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.369.530 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.372.273 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 209 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.384.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 109, total running count: 109 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.384.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.386.746 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 210 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.398.997 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 110, total running count: 110 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.399.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.401.114 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 211 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.413.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 111, total running count: 111 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.414.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.416.603 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 212 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.428.829 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 112, total running count: 112 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.429.419 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.431.163 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 213 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.443.970 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 113, total running count: 113 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.444.468 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.446.970 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 214 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.458.889 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 114, total running count: 114 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.459.440 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.461.437 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 215 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.473.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 115, total running count: 115 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.474.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.477.098 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 216 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.488.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 116, total running count: 116 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.489.394 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.491.413 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 217 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.503.922 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 117, total running count: 117 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.504.423 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.507.154 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 218 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.518.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 118, total running count: 118 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.519.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.521.810 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 219 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.533.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 119, total running count: 119 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.534.326 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.536.414 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 220 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.548.817 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 120, total running count: 120 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.549.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.551.828 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 221 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.563.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 121, total running count: 121 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.564.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.566.404 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 222 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.578.587 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 122, total running count: 122 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.579.043 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.580.997 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 223 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.593.625 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 123, total running count: 123 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.594.028 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.595.780 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 224 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.608.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 124, total running count: 124 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.608.915 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.611.345 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 225 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.623.380 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 125, total running count: 125 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.623.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.625.673 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 226 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.638.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 126, total running count: 126 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.638.973 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.641.317 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 227 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.653.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 127, total running count: 127 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.653.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.655.644 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 228 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.668.407 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 128, total running count: 128 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.668.890 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.671.506 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 229 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.683.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 129, total running count: 129 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.684.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.686.177 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 230 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.698.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 130, total running count: 130 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.699.222 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.701.825 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 231 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.713.715 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 131, total running count: 131 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.714.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.716.227 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 232 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.728.666 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 132, total running count: 132 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.729.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.731.780 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 233 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.743.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 133, total running count: 133 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.744.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.746.300 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 234 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.758.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 134, total running count: 134 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.759.181 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.761.029 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 235 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.773.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 135, total running count: 135 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.774.352 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.776.793 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 236 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.788.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 136, total running count: 136 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.789.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.791.327 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 237 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.803.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 137, total running count: 137 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.804.239 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.806.007 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 238 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.818.727 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 138, total running count: 138 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.819.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.821.599 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 239 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.833.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 139, total running count: 139 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.834.336 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.835.955 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 240 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.848.815 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 140, total running count: 140 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.849.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.851.903 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 241 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.863.752 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 141, total running count: 141 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.864.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.866.705 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 242 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.878.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 142, total running count: 142 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.884.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.886.040 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 243 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.898.681 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 143, total running count: 143 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.899.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.901.990 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 244 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.913.651 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 144, total running count: 144 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.914.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.916.596 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 245 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:29.928.680 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 145, total running count: 145 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.929.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.930.994 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 246 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.943.754 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 146, total running count: 146 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.944.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.946.461 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 247 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.958.675 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 147, total running count: 147 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:29.959.042 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.961.116 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 248 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:29.973.413 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 148, total running count: 148 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.973.969 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.975.998 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 249 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:29.988.736 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 149, total running count: 149 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:29.989.155 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:29.991.406 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 250 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.004.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 150, total running count: 150 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.004.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.006.942 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 251 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.019.551 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 151, total running count: 151 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.019.899 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.022.448 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 252 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.034.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 152, total running count: 152 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.034.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.036.717 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 253 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.049.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 153, total running count: 153 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.049.964 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.052.230 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 254 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:30.064.661 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 154, total running count: 154 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.065.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.066.596 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 255 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.079.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 155, total running count: 155 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.079.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.082.373 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 256 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.094.625 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 156, total running count: 156 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.094.968 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.097.036 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 257 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.109.552 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 157, total running count: 157 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.109.982 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.112.621 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 258 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.124.750 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 158, total running count: 158 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.125.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.127.119 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 259 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.139.637 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 159, total running count: 159 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.140.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.142.707 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 260 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.154.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 160, total running count: 160 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.154.909 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.157.521 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 261 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.169.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 161, total running count: 161 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.169.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.172.056 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 262 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.184.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 162, total running count: 162 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.184.719 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.186.487 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 263 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.199.219 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 163, total running count: 163 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.199.570 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.202.232 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 264 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.214.109 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 164, total running count: 164 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:30.214.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.216.996 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 265 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:30.229.219 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 165, total running count: 165 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:30.229.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.231.549 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 266 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.244.380 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 166, total running count: 166 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.244.737 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.247.324 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 267 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.259.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 167, total running count: 167 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:30.259.740 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.261.967 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 268 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.274.379 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 168, total running count: 168 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.274.731 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.276.653 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 269 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.289.024 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 169, total running count: 169 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.289.382 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.290.935 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 270 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.304.043 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 170, total running count: 170 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.304.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.306.531 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 271 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.318.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 171, total running count: 171 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:30.319.374 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.320.994 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 272 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.333.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 172, total running count: 172 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.334.391 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.336.900 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 273 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.348.908 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 173, total running count: 173 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.349.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.351.472 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 274 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.363.771 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 174, total running count: 174 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.364.313 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.366.892 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 275 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.378.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 175, total running count: 175 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.379.264 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.381.307 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 276 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.393.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 176, total running count: 176 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.394.178 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.395.750 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 277 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.408.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 177, total running count: 177 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.409.216 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.411.194 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 278 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.423.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 178, total running count: 178 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.424.275 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.426.792 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 279 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:30.438.609 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 179, total running count: 179 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.439.243 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.441.272 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 280 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:30.453.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 180, total running count: 180 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.454.224 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.455.982 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 281 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.468.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 181, total running count: 181 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:30.469.115 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.471.388 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 282 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.483.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 182, total running count: 182 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.484.134 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.485.757 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 283 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.498.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 183, total running count: 183 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.499.130 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.501.099 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 284 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.513.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 184, total running count: 184 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.514.120 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.516.603 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 285 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.528.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 185, total running count: 185 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:30.529.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.530.927 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 286 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:30.543.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 186, total running count: 186 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:30.543.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.546.507 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 287 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.558.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 187, total running count: 187 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.558.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.561.049 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 288 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.573.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 188, total running count: 188 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.573.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.575.753 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 289 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.588.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 189, total running count: 189 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.588.728 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.590.350 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 290 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.603.233 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 190, total running count: 190 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.603.613 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.605.733 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 291 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.618.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 191, total running count: 191 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.618.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.621.078 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 292 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.632.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 192, total running count: 192 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.633.212 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.635.549 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 293 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.647.791 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 193, total running count: 193 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.648.222 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.649.831 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 294 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:30.662.915 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 194, total running count: 194 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.663.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.665.524 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 295 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.677.833 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 195, total running count: 195 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.678.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.680.161 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 296 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.692.665 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 196, total running count: 196 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.693.111 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.695.771 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 297 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.707.708 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 197, total running count: 197 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.708.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.710.114 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 298 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.722.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 198, total running count: 198 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.723.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.725.780 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 299 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.737.443 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 199, total running count: 199 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.737.905 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.740.055 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 300 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.752.388 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 200, total running count: 200 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.752.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.755.448 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 301 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.767.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 201, total running count: 201 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.767.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.769.838 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 302 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.782.370 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 202, total running count: 202 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.782.733 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.784.273 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 303 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.797.203 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 203, total running count: 203 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.797.570 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.800.093 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 304 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.812.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 204, total running count: 204 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:30.812.560 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.814.631 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 305 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:30.827.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 205, total running count: 205 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.827.443 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.829.132 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 306 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.841.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 206, total running count: 206 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.842.434 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.844.583 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 307 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.856.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 207, total running count: 207 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:30.857.286 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.859.915 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 308 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.871.806 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 208, total running count: 208 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.872.181 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.874.375 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 309 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.886.806 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 209, total running count: 209 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:30.887.174 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.888.746 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 310 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.901.544 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 210, total running count: 210 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.902.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.904.212 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 311 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.916.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 211, total running count: 211 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.917.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.919.726 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 312 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.931.714 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 212, total running count: 212 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.932.086 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.934.423 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 313 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.946.489 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 213, total running count: 213 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:30.946.860 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.949.000 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 314 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.961.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 214, total running count: 214 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:30.961.762 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.963.678 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 315 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:30.976.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 215, total running count: 215 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:30.976.609 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.978.206 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 316 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:30.991.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 216, total running count: 216 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:30.991.601 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:30.993.719 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 317 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.005.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 217, total running count: 217 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:31.006.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.008.061 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 318 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.021.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 218, total running count: 218 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.021.653 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.023.885 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 319 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.036.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 219, total running count: 219 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.036.597 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.038.715 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 320 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.051.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 220, total running count: 220 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.051.596 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.053.239 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 321 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.065.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 221, total running count: 221 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.066.423 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.069.122 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 322 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:31.080.832 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 222, total running count: 222 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.081.267 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.083.820 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 323 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.095.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 223, total running count: 223 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.096.210 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.098.397 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 324 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.110.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 224, total running count: 224 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.111.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.112.992 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 325 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:31.125.508 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 225, total running count: 225 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.126.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.128.831 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 326 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.140.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 226, total running count: 226 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.141.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.143.359 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 327 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.155.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 227, total running count: 227 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.156.100 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.157.807 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 328 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.170.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 228, total running count: 228 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.171.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.173.125 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 329 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.185.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 229, total running count: 229 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.186.108 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.188.564 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 330 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.200.638 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 230, total running count: 230 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.201.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.203.170 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 331 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.215.459 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 231, total running count: 231 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.216.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.218.697 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 332 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.230.535 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 232, total running count: 232 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.230.925 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.233.273 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 333 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.245.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 233, total running count: 233 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.245.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.247.576 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 334 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.260.493 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 234, total running count: 234 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.260.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.263.079 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 335 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.275.348 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 235, total running count: 235 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.275.935 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.278.682 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 336 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.290.535 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 236, total running count: 236 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.291.033 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.293.407 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 337 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.305.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 237, total running count: 237 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.306.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.308.007 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 338 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.320.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 238, total running count: 238 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.321.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.323.509 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 339 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.335.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 239, total running count: 239 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.336.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.337.814 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 340 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.350.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 240, total running count: 240 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:31.351.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.353.751 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 341 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.365.660 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 241, total running count: 241 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.366.396 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.368.499 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 342 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:31.380.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 242, total running count: 242 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.381.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.383.948 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 343 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.395.872 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 243, total running count: 243 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:31.396.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.398.393 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 344 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.411.011 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 244, total running count: 244 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.411.419 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.414.050 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 345 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.425.841 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 245, total running count: 245 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.426.442 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.428.593 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 346 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.440.960 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 246, total running count: 246 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:31.441.395 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.442.992 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 347 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.455.764 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 247, total running count: 247 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.456.292 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.458.690 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 348 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.470.793 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 248, total running count: 248 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.471.176 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.473.055 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 349 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.485.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 249, total running count: 249 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.486.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.488.729 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 350 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.500.769 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 250, total running count: 250 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:31.501.213 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.503.467 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 351 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.515.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 251, total running count: 251 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.516.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.517.935 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 352 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.530.640 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 252, total running count: 252 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.531.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.533.783 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 353 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.545.922 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 253, total running count: 253 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.546.372 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.548.684 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 354 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.560.818 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 254, total running count: 254 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.561.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.563.065 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 355 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.575.751 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 255, total running count: 255 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.576.277 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.578.586 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 356 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:31.591.326 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 256, total running count: 256 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.591.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.594.250 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 357 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.606.165 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 257, total running count: 257 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.607.001 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.608.997 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 358 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.621.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 258, total running count: 258 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.622.087 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.624.776 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 359 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.636.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 259, total running count: 259 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.637.291 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.639.075 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 360 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.651.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 260, total running count: 260 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.652.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.654.413 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 361 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.666.916 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 261, total running count: 261 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.667.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.669.945 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 362 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.681.938 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 262, total running count: 262 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.682.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.684.650 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 363 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.697.071 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 263, total running count: 263 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.697.572 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.699.174 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 364 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.712.066 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 264, total running count: 264 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.712.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.714.969 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 365 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.727.051 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 265, total running count: 265 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.727.657 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.729.311 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 366 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.742.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 266, total running count: 266 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.742.598 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.745.045 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 367 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.757.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 267, total running count: 267 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.757.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.759.620 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 368 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.772.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 268, total running count: 268 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.772.583 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.774.114 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 369 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.787.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 269, total running count: 269 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.787.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.789.796 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 370 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.802.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 270, total running count: 270 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:31.802.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.804.380 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 371 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:31.816.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 271, total running count: 271 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.817.515 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.819.080 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 372 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.831.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 272, total running count: 272 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.832.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.834.658 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 373 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:31.846.933 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 273, total running count: 273 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.847.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.850.126 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 374 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.861.948 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 274, total running count: 274 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.862.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.864.681 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 375 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.876.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 275, total running count: 275 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.877.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.880.225 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 376 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.891.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 276, total running count: 276 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.892.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.895.010 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 377 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.906.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 277, total running count: 277 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.907.521 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.909.504 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 378 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.922.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 278, total running count: 278 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.922.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.924.908 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 379 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.937.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 279, total running count: 279 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.937.793 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.939.505 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 380 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.952.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 280, total running count: 280 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.952.756 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.955.192 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 381 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.967.202 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 281, total running count: 281 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:31.967.766 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.969.844 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 382 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.982.183 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 282, total running count: 282 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:31.982.622 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:31.984.423 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 383 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:31.997.105 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 283, total running count: 283 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:31.997.602 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.000.246 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 384 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.012.124 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 284, total running count: 284 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.012.590 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.014.896 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 385 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.027.250 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 285, total running count: 285 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.027.589 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.029.384 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 386 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.041.989 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 286, total running count: 286 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.042.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.044.126 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 387 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.057.051 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 287, total running count: 287 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.057.421 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.059.852 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 388 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.071.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 288, total running count: 288 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.072.326 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.074.597 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 389 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.086.948 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 289, total running count: 289 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.087.308 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.089.307 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 390 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.101.969 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 290, total running count: 290 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.102.325 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.103.888 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 391 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.116.721 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 291, total running count: 291 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.117.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.119.761 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 392 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:32.131.638 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 292, total running count: 292 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.131.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.134.252 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 393 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.146.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 293, total running count: 293 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.146.877 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.148.670 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 394 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.161.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 294, total running count: 294 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.161.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.164.191 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 395 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.176.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 295, total running count: 295 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.176.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.178.827 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 396 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.191.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 296, total running count: 296 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.191.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.193.527 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 397 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.206.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 297, total running count: 297 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.206.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.208.940 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 398 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.221.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 298, total running count: 298 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.221.522 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.223.268 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 399 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.236.199 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 299, total running count: 299 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.236.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.238.800 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 400 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.251.213 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 300, total running count: 300 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.251.576 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.253.115 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 401 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.266.239 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 301, total running count: 301 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.266.599 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.268.613 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 402 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.281.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 302, total running count: 302 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.281.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.284.003 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 403 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.296.347 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 303, total running count: 303 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.296.717 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.298.319 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 404 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.311.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 304, total running count: 304 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.311.716 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.313.679 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 405 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.326.269 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 305, total running count: 305 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.326.641 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.328.158 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 406 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.341.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 306, total running count: 306 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.341.649 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.343.872 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 407 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.356.146 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 307, total running count: 307 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.356.525 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.358.444 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 408 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.371.096 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 308, total running count: 308 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.371.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.373.995 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 409 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.386.115 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 309, total running count: 309 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.386.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.388.539 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 410 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:32.401.053 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 310, total running count: 310 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.401.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.403.146 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 411 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:32.415.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 311, total running count: 311 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.416.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.418.927 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 412 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.430.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 312, total running count: 312 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.431.354 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.433.341 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 413 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.445.711 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 313, total running count: 313 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.446.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.448.042 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 414 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.460.840 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 314, total running count: 314 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.461.278 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.463.913 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 415 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:32.475.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 315, total running count: 315 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:32.476.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.478.281 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 416 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.490.701 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 316, total running count: 316 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.491.349 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.493.822 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 417 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.505.831 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 317, total running count: 317 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.506.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.507.929 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 418 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.520.688 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 318, total running count: 318 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.521.156 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.523.385 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 419 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.535.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 319, total running count: 319 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.536.148 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.538.827 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 420 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.550.582 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 320, total running count: 320 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.551.009 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.552.980 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 421 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.565.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 321, total running count: 321 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.566.105 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.568.631 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 422 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.580.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 322, total running count: 322 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:32.581.139 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.583.088 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 423 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.595.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 323, total running count: 323 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.596.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.597.669 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 424 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.610.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 324, total running count: 324 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.610.966 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.613.555 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 425 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.625.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 325, total running count: 325 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.625.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.627.793 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 426 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.640.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 326, total running count: 326 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.641.184 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.643.852 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 427 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.655.922 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 327, total running count: 327 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.656.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.658.677 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 428 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.671.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 328, total running count: 328 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.671.926 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.674.512 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 429 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.686.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 329, total running count: 329 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.687.244 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.689.301 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 430 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.702.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 330, total running count: 330 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:32.702.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.704.148 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 431 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:32.717.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 331, total running count: 331 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.717.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.720.168 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 432 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.732.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 332, total running count: 332 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:32.732.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.735.046 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 433 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.747.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 333, total running count: 333 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.747.859 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.749.852 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 434 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:32.762.468 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 334, total running count: 334 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.763.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.766.137 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 435 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.778.033 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 335, total running count: 335 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.778.389 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.780.910 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 436 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:32.792.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 336, total running count: 336 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.793.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.795.597 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 437 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.808.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 337, total running count: 337 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.808.642 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.811.377 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 438 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.823.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 338, total running count: 338 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.823.815 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.826.294 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 439 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:32.838.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 339, total running count: 339 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:32.839.146 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.841.307 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 440 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.853.860 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 340, total running count: 340 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.854.214 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.856.234 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 441 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.868.987 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 341, total running count: 341 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.869.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.871.182 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 442 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.884.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 342, total running count: 342 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:32.884.412 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.886.036 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 443 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.899.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 343, total running count: 343 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:32.899.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.901.928 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 444 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.914.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 344, total running count: 344 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:32.914.574 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.916.662 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 445 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.929.217 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 345, total running count: 345 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:32.929.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.931.675 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 446 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:32.944.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 346, total running count: 346 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:32.944.703 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.946.397 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 447 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:32.959.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 347, total running count: 347 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:32.959.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.962.384 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 448 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.974.668 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 348, total running count: 348 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:32.975.027 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.977.095 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 449 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:32.989.832 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 349, total running count: 349 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:32.990.174 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:32.992.754 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 450 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:33.004.744 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 350, total running count: 350 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.005.132 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.007.547 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 451 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.020.045 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 351, total running count: 351 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.020.387 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.022.296 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 452 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:33.035.285 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 352, total running count: 352 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.035.637 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.038.226 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 453 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:33.050.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 353, total running count: 353 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:33.050.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.052.699 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 454 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.065.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 354, total running count: 354 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.065.765 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.068.418 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 455 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.080.651 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 355, total running count: 355 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.080.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.082.845 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 456 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.095.535 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 356, total running count: 356 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.095.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.098.425 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 457 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:33.110.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 357, total running count: 357 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.111.042 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.112.760 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 458 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.125.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 358, total running count: 358 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:33.126.173 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.128.168 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 459 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.140.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 359, total running count: 359 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.141.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.143.798 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 460 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.155.611 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 360, total running count: 360 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.155.944 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.158.125 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 461 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.170.795 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 361, total running count: 361 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:33.171.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.172.936 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 462 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:33.185.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 362, total running count: 362 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.186.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.188.893 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 463 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.200.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 363, total running count: 363 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:33.201.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.203.445 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 464 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.215.820 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 364, total running count: 364 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.216.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.218.014 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 465 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.230.728 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 365, total running count: 365 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:33.231.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.232.723 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 466 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.246.002 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 366, total running count: 366 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:33.246.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.248.608 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 467 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:33.261.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 367, total running count: 367 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.261.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.263.140 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 468 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.275.966 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 368, total running count: 368 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.276.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.278.771 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 469 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.290.955 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 369, total running count: 369 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.291.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.293.478 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 470 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.305.948 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 370, total running count: 370 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.306.290 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.308.160 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 471 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.320.988 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 371, total running count: 371 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.321.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.323.110 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 472 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:33.336.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 372, total running count: 372 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.336.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.338.709 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 473 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:33.351.146 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 373, total running count: 373 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:33.351.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.353.303 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 474 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.365.997 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 374, total running count: 374 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.366.356 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.368.947 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 475 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.380.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 375, total running count: 375 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.381.250 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.383.485 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 476 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.395.871 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 376, total running count: 376 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.396.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.398.020 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 477 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.410.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 377, total running count: 377 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.411.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.413.820 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 478 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.425.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 378, total running count: 378 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.426.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.428.289 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 479 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.440.832 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 379, total running count: 379 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:33.441.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.443.941 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 480 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.455.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 380, total running count: 380 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.456.146 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.458.249 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 481 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:33.470.815 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 381, total running count: 381 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:33.471.154 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.473.948 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 482 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.485.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 382, total running count: 382 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.486.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.488.194 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 483 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.500.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 383, total running count: 383 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.501.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.502.736 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 484 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.515.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 384, total running count: 384 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.515.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.518.303 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 485 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.530.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 385, total running count: 385 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:33.530.674 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.532.714 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 486 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:33.545.238 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 386, total running count: 386 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:33.545.580 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.547.341 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 487 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.560.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 387, total running count: 387 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.560.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.562.148 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 488 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:33.574.962 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 388, total running count: 388 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.575.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.578.062 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 489 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:33.590.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 389, total running count: 389 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.590.400 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.592.767 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 490 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:33.604.829 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 390, total running count: 390 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:33.605.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.607.495 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 491 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.619.780 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 391, total running count: 391 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.620.147 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.621.874 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 492 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.634.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 392, total running count: 392 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.635.218 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.637.798 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 493 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.649.935 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 393, total running count: 393 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.650.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.652.531 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 494 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:33.664.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 394, total running count: 394 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.665.114 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.667.277 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 495 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:33.679.769 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 395, total running count: 395 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.680.128 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.681.767 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 496 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.694.818 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 396, total running count: 396 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:33.695.154 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.697.359 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 497 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.709.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 397, total running count: 397 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.710.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.712.834 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 498 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:33.724.718 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 398, total running count: 398 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.725.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.727.273 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 499 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.739.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 399, total running count: 399 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:33.739.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.741.752 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 500 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.754.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 400, total running count: 400 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.754.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.756.501 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 501 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.769.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 401, total running count: 401 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:33.769.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.772.345 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 502 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.784.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 402, total running count: 402 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.784.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.786.870 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 503 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.799.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 403, total running count: 403 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.799.795 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.802.507 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 504 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.814.395 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 404, total running count: 404 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.814.740 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.816.909 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 505 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.829.349 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 405, total running count: 405 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.829.709 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.831.491 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 506 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.844.195 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 406, total running count: 406 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.844.525 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.846.231 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 507 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.859.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 407, total running count: 407 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:33.859.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.861.934 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 508 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.874.015 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 408, total running count: 408 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.874.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.875.941 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 509 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.888.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 409, total running count: 409 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.889.249 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.891.662 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 510 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.903.818 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 410, total running count: 410 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.904.148 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.906.255 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 511 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.918.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 411, total running count: 411 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:33.918.943 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.921.246 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 512 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:33.933.516 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 412, total running count: 412 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.933.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.935.781 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 513 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.948.482 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 413, total running count: 413 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.948.808 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.951.449 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 514 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.963.354 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 414, total running count: 414 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:33.963.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.965.866 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 515 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:33.978.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 415, total running count: 415 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:33.978.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.980.566 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 516 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.993.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 416, total running count: 416 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:33.993.501 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:33.996.260 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 517 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.007.989 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 417, total running count: 417 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.008.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.010.721 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 518 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:34.023.154 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 418, total running count: 418 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.023.479 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.025.182 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 519 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.038.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 419, total running count: 419 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.038.587 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.041.008 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 520 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:34.053.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 420, total running count: 420 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.053.444 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.055.447 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 521 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:34.068.011 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 421, total running count: 421 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.068.346 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.069.898 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 522 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:34.082.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 422, total running count: 422 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.083.107 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.085.948 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 523 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.097.844 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 423, total running count: 423 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.098.191 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.100.570 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 524 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.112.768 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 424, total running count: 424 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.113.102 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.115.001 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 525 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.127.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 425, total running count: 425 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.128.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.130.440 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 526 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.142.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 426, total running count: 426 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:34.143.270 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.144.893 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 527 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.157.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 427, total running count: 427 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:34.158.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.160.549 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 528 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.172.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 428, total running count: 428 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.172.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.175.444 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 529 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.187.531 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 429, total running count: 429 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.187.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.190.268 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 530 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.202.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 430, total running count: 430 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.202.856 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.204.672 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 531 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.217.481 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 431, total running count: 431 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.217.841 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.220.210 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 532 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.232.278 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 432, total running count: 432 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.232.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.234.690 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 533 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.247.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 433, total running count: 433 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.247.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.249.337 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 534 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:34.262.392 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 434, total running count: 434 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.262.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.265.051 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 535 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.277.324 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 435, total running count: 435 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.277.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.279.552 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 536 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.292.225 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 436, total running count: 436 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.292.576 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.295.093 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 537 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.307.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 437, total running count: 437 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.307.511 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.309.758 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 538 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.322.031 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 438, total running count: 438 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.322.361 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.324.542 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 539 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.336.970 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 439, total running count: 439 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:34.337.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.338.974 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 540 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.352.037 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 440, total running count: 440 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.352.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.354.619 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 541 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.366.973 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 441, total running count: 441 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.367.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.368.947 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 542 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.381.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 442, total running count: 442 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.382.250 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.384.623 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 543 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.396.716 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 443, total running count: 443 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.397.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.399.258 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 544 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.411.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 444, total running count: 444 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.411.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.414.178 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 545 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.426.627 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 445, total running count: 445 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.426.969 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.428.633 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 546 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.441.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 446, total running count: 446 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.441.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.444.148 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 547 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:34.456.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 447, total running count: 447 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.456.880 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.458.781 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 548 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.471.382 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 448, total running count: 448 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.471.736 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.473.493 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 549 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.486.232 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 449, total running count: 449 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.486.574 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.488.255 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 550 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.501.270 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 450, total running count: 450 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.501.618 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.504.020 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 551 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.516.313 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 451, total running count: 451 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.516.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.518.515 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 552 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.531.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 452, total running count: 452 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.531.650 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.534.177 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 553 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:34.546.345 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 453, total running count: 453 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.546.681 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.548.456 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 554 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.561.239 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 454, total running count: 454 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.561.592 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.564.292 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 555 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.576.269 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 455, total running count: 455 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.576.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.578.801 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 556 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.591.225 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 456, total running count: 456 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.591.589 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.593.214 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 557 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:34.606.362 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 457, total running count: 457 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.606.698 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.608.923 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 558 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.621.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 458, total running count: 458 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.621.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.623.636 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 559 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.636.375 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 459, total running count: 459 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:34.636.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.638.293 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 560 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:34.651.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 460, total running count: 460 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.651.532 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.654.114 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 561 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:34.666.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 461, total running count: 461 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.666.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.668.627 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 562 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.681.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 462, total running count: 462 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.681.404 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.683.106 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 563 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:34.695.799 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 463, total running count: 463 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.696.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.698.825 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 564 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.710.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 464, total running count: 464 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.710.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.713.416 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 565 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.725.440 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 465, total running count: 465 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.725.845 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.727.866 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 566 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.740.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 466, total running count: 466 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.740.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.742.577 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 567 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.755.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 467, total running count: 467 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.755.416 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.757.067 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 568 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.770.089 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 468, total running count: 468 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:40:34.770.483 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_1 execution count: 1, execution time: 182739 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:40:34.770.653 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_1 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:40:34.770.848 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty epoch: 1 step: 468, loss is 2.3020758628845215 [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:40:34.771.958 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.h:76] ContinueSend] continue send at the beginning of the epoch [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:40:34.773.337 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:40:34.773.399 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:40:34.773.440 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_1 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.773.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.776.380 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 569 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.788.514 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 1, total running count: 469 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.788.876 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.790.784 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 570 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:34.803.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 2, total running count: 470 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.803.737 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.806.407 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 571 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:34.818.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 3, total running count: 471 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.818.694 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.820.935 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 572 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.833.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 4, total running count: 472 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.833.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.835.401 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 573 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.848.378 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 5, total running count: 473 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.848.718 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.851.249 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 574 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.863.336 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 6, total running count: 474 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.863.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.865.902 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 575 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.878.102 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 7, total running count: 475 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.878.441 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.880.240 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 576 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:34.893.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 8, total running count: 476 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.893.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.895.987 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 577 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.907.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 9, total running count: 477 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:34.908.295 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.910.452 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 578 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.922.788 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 10, total running count: 478 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.923.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.925.034 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 579 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.937.883 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 11, total running count: 479 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.938.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.940.742 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 580 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.952.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 12, total running count: 480 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.953.275 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.955.294 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 581 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:34.968.016 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 13, total running count: 481 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.968.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.971.055 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 582 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:34.982.962 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 14, total running count: 482 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:34.983.304 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:34.985.561 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 583 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:34.997.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 15, total running count: 483 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:34.998.146 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.000.065 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 584 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.012.601 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 16, total running count: 484 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.012.930 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.014.604 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 585 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.027.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 17, total running count: 485 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.027.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.030.636 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 586 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.042.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 18, total running count: 486 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.042.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.045.108 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 587 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.057.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 19, total running count: 487 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.057.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.059.529 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 588 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.072.036 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 20, total running count: 488 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.072.392 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.074.955 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 589 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.087.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 21, total running count: 489 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.087.544 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.089.279 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 590 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.102.051 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 22, total running count: 490 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.102.395 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.104.698 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 591 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.116.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 23, total running count: 491 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.117.001 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.118.974 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 592 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.131.462 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 24, total running count: 492 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.131.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.134.416 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 593 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.146.552 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 25, total running count: 493 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.146.885 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.148.964 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 594 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.161.480 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 26, total running count: 494 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.163.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.165.863 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 595 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.177.580 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 27, total running count: 495 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.178.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.181.520 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 596 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.193.583 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 28, total running count: 496 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.194.015 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.196.036 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 597 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.208.568 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 29, total running count: 497 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.208.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.211.733 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 598 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.223.544 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 30, total running count: 498 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.223.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.226.599 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 599 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.238.254 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 31, total running count: 499 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.238.594 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.241.290 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 600 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.253.099 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 32, total running count: 500 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.253.460 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.255.517 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 601 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.268.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 33, total running count: 501 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.268.433 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.271.029 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 602 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.283.009 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 34, total running count: 502 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.283.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.285.373 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 603 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.298.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 35, total running count: 503 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.298.503 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.300.918 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 604 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.313.061 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 36, total running count: 504 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.313.517 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.315.574 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 605 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.328.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 37, total running count: 505 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.328.442 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.330.245 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 606 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.343.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 38, total running count: 506 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.343.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.345.873 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 607 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.357.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 39, total running count: 507 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.358.182 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.360.514 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 608 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.372.559 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 40, total running count: 508 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.373.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.374.934 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 609 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.387.703 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 41, total running count: 509 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.388.039 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.390.569 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 610 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.402.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 42, total running count: 510 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.402.882 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.405.315 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 611 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.417.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 43, total running count: 511 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.418.154 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.420.751 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 612 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.432.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 44, total running count: 512 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.432.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.435.352 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 613 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.448.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 45, total running count: 513 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.449.352 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.451.283 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 614 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.464.503 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 46, total running count: 514 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.464.827 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.467.083 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 615 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.479.532 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 47, total running count: 515 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.479.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.481.524 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 616 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.494.558 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 48, total running count: 516 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.494.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.496.837 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 617 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.509.375 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 49, total running count: 517 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.509.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.512.436 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 618 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.524.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 50, total running count: 518 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.524.675 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.526.992 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 619 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.539.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 51, total running count: 519 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.539.853 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.542.669 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 620 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.554.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 52, total running count: 520 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.555.105 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.557.296 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 621 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.569.992 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 53, total running count: 521 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.570.354 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.572.854 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 622 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.585.031 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 54, total running count: 522 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.585.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.587.124 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 623 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.600.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 55, total running count: 523 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.600.559 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.602.726 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 624 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.615.158 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 56, total running count: 524 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.615.703 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.618.360 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 625 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.630.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 57, total running count: 525 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.630.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.633.038 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 626 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.645.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 58, total running count: 526 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.645.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.647.571 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 627 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.660.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 59, total running count: 527 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.660.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.663.008 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 628 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.675.246 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 60, total running count: 528 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.675.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.677.440 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 629 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.690.498 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 61, total running count: 529 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.690.882 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.693.246 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 630 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.706.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 62, total running count: 530 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.706.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.709.340 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 631 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.721.399 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 63, total running count: 531 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.721.765 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.723.778 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 632 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.736.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 64, total running count: 532 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.736.806 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.738.367 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 633 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.751.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 65, total running count: 533 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.751.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.754.275 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 634 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.766.587 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 66, total running count: 534 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.766.946 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.768.585 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 635 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.781.558 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 67, total running count: 535 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.784.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.786.289 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 636 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.799.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 68, total running count: 536 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.800.993 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.802.799 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 637 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.815.609 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 69, total running count: 537 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.816.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.818.254 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 638 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.834.192 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 70, total running count: 538 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.834.570 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.836.235 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 639 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.849.969 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 71, total running count: 539 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.850.336 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.852.973 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 640 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:35.865.187 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 72, total running count: 540 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.865.530 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.867.673 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 641 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.880.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 73, total running count: 541 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:35.880.833 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.883.470 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 642 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.895.601 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 74, total running count: 542 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.895.955 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.897.961 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 643 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.910.858 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 75, total running count: 543 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:35.911.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.913.611 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 644 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.926.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 76, total running count: 544 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.926.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.928.168 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 645 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.941.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 77, total running count: 545 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.941.526 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.944.061 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 646 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.960.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 78, total running count: 546 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.960.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.963.258 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 647 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.975.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 79, total running count: 547 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.976.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.977.849 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 648 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:35.990.879 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 80, total running count: 548 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:35.991.218 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:35.993.561 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 649 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.005.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 81, total running count: 549 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.006.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.007.955 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 650 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.020.885 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 82, total running count: 550 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.021.235 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.023.462 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 651 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.035.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 83, total running count: 551 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.036.344 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.038.956 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 652 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.051.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 84, total running count: 552 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.051.378 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.053.275 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 653 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.066.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 85, total running count: 553 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.066.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.068.751 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 654 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.081.099 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 86, total running count: 554 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.081.441 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.084.841 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 655 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.098.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 87, total running count: 555 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.098.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.100.680 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 656 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.113.219 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 88, total running count: 556 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.113.559 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.115.190 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 657 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.128.385 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 89, total running count: 557 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.128.727 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.130.991 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 658 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.143.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 90, total running count: 558 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.143.782 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.145.599 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 659 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.158.501 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 91, total running count: 559 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.158.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.161.385 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 660 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.173.590 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 92, total running count: 560 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.173.970 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.175.932 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 661 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.188.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 93, total running count: 561 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.188.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.191.495 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 662 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.203.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 94, total running count: 562 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.203.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.206.220 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 663 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.218.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 95, total running count: 563 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.219.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.222.084 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 664 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.233.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 96, total running count: 564 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.234.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.236.496 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 665 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.249.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 97, total running count: 565 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.249.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.251.970 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 666 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.264.114 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 98, total running count: 566 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.264.465 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.266.690 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 667 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.279.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 99, total running count: 567 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.279.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.281.305 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 668 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.294.089 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 100, total running count: 568 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.294.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.296.671 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 669 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.309.087 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 101, total running count: 569 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.309.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.312.112 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 670 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.324.187 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 102, total running count: 570 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.324.517 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.326.573 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 671 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.339.235 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 103, total running count: 571 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.339.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.342.111 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 672 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.354.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 104, total running count: 572 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.354.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.356.614 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 673 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.369.362 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 105, total running count: 573 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.369.739 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.371.331 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 674 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.384.336 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 106, total running count: 574 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.384.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.386.926 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 675 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.399.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 107, total running count: 575 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.399.632 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.401.245 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 676 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.414.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 108, total running count: 576 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.414.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.416.739 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 677 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.429.650 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 109, total running count: 577 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.430.949 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.433.542 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 678 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.445.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 110, total running count: 578 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.446.746 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.449.329 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 679 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.461.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 111, total running count: 579 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.462.627 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.464.937 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 680 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.477.241 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 112, total running count: 580 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.478.205 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.480.533 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 681 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.492.756 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 113, total running count: 581 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.493.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.495.854 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 682 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.508.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 114, total running count: 582 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.514.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.516.101 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 683 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.528.935 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 115, total running count: 583 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.531.114 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.532.830 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 684 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.545.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 116, total running count: 584 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.546.820 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.549.337 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 685 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.561.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 117, total running count: 585 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.562.516 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.565.258 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 686 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.577.202 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 118, total running count: 586 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.578.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.579.950 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 687 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.592.992 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 119, total running count: 587 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.593.816 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.595.462 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 688 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.608.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 120, total running count: 588 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.609.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.612.044 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 689 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.624.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 121, total running count: 589 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.625.425 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.627.553 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 690 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.640.169 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 122, total running count: 590 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.641.078 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.642.946 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 691 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.655.805 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 123, total running count: 591 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.656.948 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.659.521 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 692 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.671.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 124, total running count: 592 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.672.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.674.929 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 693 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.687.065 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 125, total running count: 593 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.688.169 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.690.407 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 694 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.702.978 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 126, total running count: 594 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.703.860 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.706.017 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 695 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.718.466 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 127, total running count: 595 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.719.567 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.721.586 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 696 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.734.164 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 128, total running count: 596 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.736.496 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.738.258 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 697 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.751.217 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 129, total running count: 597 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.751.973 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.753.871 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 698 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.766.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 130, total running count: 598 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.767.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.769.835 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 699 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.782.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 131, total running count: 599 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.783.455 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.785.403 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 700 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.798.177 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 132, total running count: 600 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.799.192 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.800.898 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 701 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.813.897 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 133, total running count: 601 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.814.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.817.591 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 702 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.829.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 134, total running count: 602 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.830.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.833.081 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 703 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.845.132 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 135, total running count: 603 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.848.840 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.850.851 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 704 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.863.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 136, total running count: 604 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.864.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.867.460 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 705 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.879.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 137, total running count: 605 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.881.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.883.074 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 706 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.895.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 138, total running count: 606 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.896.720 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.898.804 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 707 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.911.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 139, total running count: 607 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:36.912.515 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.914.727 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 708 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.926.942 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 140, total running count: 608 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.928.389 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.930.179 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 709 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:36.943.267 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 141, total running count: 609 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.949.140 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.951.385 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 710 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.963.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 142, total running count: 610 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.964.584 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.966.735 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 711 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:36.979.277 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 143, total running count: 611 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.980.285 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.982.191 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 712 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:36.995.002 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 144, total running count: 612 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:36.995.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:36.997.890 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 713 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.010.748 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 145, total running count: 613 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.011.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.013.476 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 714 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:37.026.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 146, total running count: 614 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:37.027.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.029.300 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 715 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.042.150 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 147, total running count: 615 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.043.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.044.663 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 716 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.057.883 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 148, total running count: 616 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.058.778 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.061.408 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 717 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.073.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 149, total running count: 617 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.074.599 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.076.836 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 718 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.089.208 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 150, total running count: 618 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.090.556 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.092.215 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 719 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:37.105.353 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 151, total running count: 619 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:37.106.576 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.108.887 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 720 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.121.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 152, total running count: 620 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.122.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.124.504 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 721 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.137.156 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 153, total running count: 621 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.138.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.139.974 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 722 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.152.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 154, total running count: 622 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.153.897 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.155.571 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 723 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.168.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 155, total running count: 623 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:37.169.785 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.172.153 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 724 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.184.408 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 156, total running count: 624 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.185.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.186.791 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 725 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.199.864 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 157, total running count: 625 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.200.693 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.203.312 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 726 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:37.215.458 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 158, total running count: 626 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.216.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.218.821 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 727 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.231.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 159, total running count: 627 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.232.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.234.397 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 728 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.246.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 160, total running count: 628 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.247.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.249.900 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 729 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.262.346 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 161, total running count: 629 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:37.263.482 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.265.365 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 730 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.278.212 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 162, total running count: 630 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.278.850 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.281.005 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 731 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.293.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 163, total running count: 631 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.294.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.296.927 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 732 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.309.241 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 164, total running count: 632 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.310.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.312.503 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 733 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:37.324.833 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 165, total running count: 633 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.325.922 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.328.057 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 734 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.340.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 166, total running count: 634 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.341.644 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.343.832 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 735 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.356.184 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 167, total running count: 635 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.357.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.359.693 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 736 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.371.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 168, total running count: 636 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.372.844 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.375.312 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 737 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:37.387.545 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 169, total running count: 637 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.388.489 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.391.067 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 738 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:37.402.966 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 170, total running count: 638 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.404.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.406.978 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 739 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.419.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 171, total running count: 639 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.420.210 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.422.588 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 740 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:37.434.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 172, total running count: 640 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.435.804 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.438.371 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 741 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.450.375 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 173, total running count: 641 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.451.675 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.454.290 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 742 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.466.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 174, total running count: 642 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.467.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.469.957 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 743 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.482.325 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 175, total running count: 643 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.483.295 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.485.768 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 744 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.497.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 176, total running count: 644 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.498.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.500.535 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 745 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.513.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 177, total running count: 645 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.514.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.516.098 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 746 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.529.065 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 178, total running count: 646 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:37.529.822 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.531.480 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 747 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.544.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 179, total running count: 647 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.545.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.547.066 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 748 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.559.967 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 180, total running count: 648 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:37.560.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.562.564 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 749 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.575.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 181, total running count: 649 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.576.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.578.117 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 750 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.590.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 182, total running count: 650 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.591.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.593.681 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 751 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.606.683 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 183, total running count: 651 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.607.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.609.407 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 752 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.622.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 184, total running count: 652 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.623.331 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.625.093 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 753 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:37.637.779 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 185, total running count: 653 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:37.639.078 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.641.765 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 754 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.653.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 186, total running count: 654 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.654.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.656.513 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 755 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.669.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 187, total running count: 655 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.670.898 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.673.423 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 756 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.685.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 188, total running count: 656 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.686.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.688.021 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 757 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.700.970 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 189, total running count: 657 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.702.042 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.703.953 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 758 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.716.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 190, total running count: 658 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.717.442 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.719.416 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 759 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.732.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 191, total running count: 659 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.733.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.735.028 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 760 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.747.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 192, total running count: 660 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.748.642 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.750.563 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 761 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.763.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 193, total running count: 661 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.764.212 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.766.588 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 762 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.778.725 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 194, total running count: 662 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.779.878 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.782.270 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 763 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.794.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 195, total running count: 663 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.795.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.797.944 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 764 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.810.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 196, total running count: 664 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.811.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.813.864 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 765 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.825.988 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 197, total running count: 665 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.827.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.829.653 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 766 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.841.598 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 198, total running count: 666 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.842.521 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.845.221 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 767 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.857.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 199, total running count: 667 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:37.858.207 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.860.767 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 768 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.872.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 200, total running count: 668 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:37.873.966 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.876.550 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 769 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.888.615 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 201, total running count: 669 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.889.806 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.892.331 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 770 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.904.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 202, total running count: 670 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.905.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.906.854 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 771 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.920.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 203, total running count: 671 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.921.232 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.923.752 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 772 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:37.935.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 204, total running count: 672 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.936.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.938.540 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 773 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.951.560 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 205, total running count: 673 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:37.952.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.954.488 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 774 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.967.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 206, total running count: 674 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.967.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.970.204 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 775 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:37.982.777 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 207, total running count: 675 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:37.983.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:37.986.155 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 776 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:37.998.129 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 208, total running count: 676 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:37.999.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.001.745 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 777 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.013.818 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 209, total running count: 677 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.014.850 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.017.283 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 778 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.029.527 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 210, total running count: 678 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.030.619 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.033.038 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 779 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.045.370 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 211, total running count: 679 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.046.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.048.805 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 780 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.061.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 212, total running count: 680 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.061.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.063.311 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 781 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.076.270 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 213, total running count: 681 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.077.238 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.078.876 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 782 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.091.840 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 214, total running count: 682 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.092.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.095.531 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 783 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.107.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 215, total running count: 683 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.108.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.111.131 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 784 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.123.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 216, total running count: 684 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.124.325 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.126.932 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 785 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.138.947 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 217, total running count: 685 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.140.007 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.141.773 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 786 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.154.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 218, total running count: 686 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.155.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.157.264 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 787 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.170.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 219, total running count: 687 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:38.171.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.173.891 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 788 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.185.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 220, total running count: 688 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.186.657 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.188.250 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 789 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.201.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 221, total running count: 689 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.202.380 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.203.994 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 790 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.217.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 222, total running count: 690 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.217.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.219.823 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 791 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.232.744 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 223, total running count: 691 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.233.575 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.235.275 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 792 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.248.216 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 224, total running count: 692 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.249.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.250.805 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 793 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.263.583 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 225, total running count: 693 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.264.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.266.285 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 794 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.279.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 226, total running count: 694 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.279.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.281.841 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 795 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.294.661 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 227, total running count: 695 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.295.663 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.297.450 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 796 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.310.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 228, total running count: 696 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:38.311.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.313.243 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 797 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.326.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 229, total running count: 697 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.326.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.329.123 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 798 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.341.203 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 230, total running count: 698 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.342.291 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.344.894 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 799 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.357.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 231, total running count: 699 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.357.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.360.632 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 800 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.372.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 232, total running count: 700 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:38.373.269 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.374.940 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 801 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.387.899 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 233, total running count: 701 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:38.388.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.391.514 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 802 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.403.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 234, total running count: 702 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.404.501 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.406.967 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 803 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.419.207 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 235, total running count: 703 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:38.419.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.422.349 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 804 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.434.678 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 236, total running count: 704 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.435.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.437.911 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 805 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:38.449.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 237, total running count: 705 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:38.450.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.452.485 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 806 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.465.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 238, total running count: 706 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:38.466.203 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.467.988 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 807 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.480.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 239, total running count: 707 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.481.888 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.483.862 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 808 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.496.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 240, total running count: 708 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.497.349 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.499.433 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 809 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.512.106 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 241, total running count: 709 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.512.908 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.515.048 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 810 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.527.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 242, total running count: 710 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.528.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.530.861 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 811 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.543.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 243, total running count: 711 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.543.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.545.582 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 812 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.558.551 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 244, total running count: 712 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.559.618 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.562.304 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 813 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:38.574.350 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 245, total running count: 713 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.575.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.577.803 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 814 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.590.009 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 246, total running count: 714 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.590.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.593.304 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 815 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.605.620 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 247, total running count: 715 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.606.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.608.843 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 816 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.621.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 248, total running count: 716 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:38.622.259 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.624.602 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 817 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.637.027 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 249, total running count: 717 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.637.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.640.466 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 818 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:38.652.651 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 250, total running count: 718 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.653.681 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.656.058 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 819 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:38.668.439 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 251, total running count: 719 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.669.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.671.556 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 820 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.683.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 252, total running count: 720 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.684.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.686.908 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 821 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.699.689 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 253, total running count: 721 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.700.382 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.702.389 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 822 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.715.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 254, total running count: 722 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.715.828 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.717.985 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 823 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.730.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 255, total running count: 723 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:38.731.248 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.733.766 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 824 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.745.961 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 256, total running count: 724 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.746.590 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.748.358 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 825 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:38.761.259 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 257, total running count: 725 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.762.089 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.764.000 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 826 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.776.701 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 258, total running count: 726 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.777.329 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.779.475 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 827 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:38.792.091 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 259, total running count: 727 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.792.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.794.927 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 828 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.807.714 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 260, total running count: 728 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.808.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.810.351 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 829 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:38.823.225 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 261, total running count: 729 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.823.882 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.825.787 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 830 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.838.563 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 262, total running count: 730 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.839.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.841.324 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 831 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.854.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 263, total running count: 731 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.855.072 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.857.075 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 832 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.869.815 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 264, total running count: 732 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.870.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.873.069 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 833 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:38.885.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 265, total running count: 733 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.885.970 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.888.569 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 834 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.900.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 266, total running count: 734 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.901.338 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.903.991 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 835 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.916.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 267, total running count: 735 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.917.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.919.423 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 836 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.931.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 268, total running count: 736 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.932.478 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.934.903 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 837 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.947.140 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 269, total running count: 737 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:38.948.226 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.950.347 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 838 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:38.962.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 270, total running count: 738 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:38.963.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.965.957 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 839 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.978.459 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 271, total running count: 739 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:38.979.250 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.981.849 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 840 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:38.993.713 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 272, total running count: 740 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:38.994.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:38.996.523 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 841 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.009.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 273, total running count: 741 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:39.010.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.013.237 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 842 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.025.361 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 274, total running count: 742 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.026.112 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.028.613 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 843 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.040.804 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 275, total running count: 743 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:39.041.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.044.284 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 844 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.056.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 276, total running count: 744 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.057.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.059.006 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 845 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.071.968 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 277, total running count: 745 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.072.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.074.900 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 846 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.087.608 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 278, total running count: 746 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.088.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.090.617 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 847 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.103.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 279, total running count: 747 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.104.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.106.577 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 848 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:39.118.720 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 280, total running count: 748 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:39.119.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.122.149 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 849 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.134.358 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 281, total running count: 749 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.135.224 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.137.775 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 850 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.149.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 282, total running count: 750 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.150.711 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.153.279 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 851 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.165.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 283, total running count: 751 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.166.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.167.891 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 852 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:39.180.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 284, total running count: 752 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.181.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.183.301 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 853 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.196.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 285, total running count: 753 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:39.196.987 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.199.284 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 854 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.211.677 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 286, total running count: 754 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.212.432 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.214.930 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 855 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:39.227.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 287, total running count: 755 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.228.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.230.569 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 856 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.242.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 288, total running count: 756 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:39.243.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.245.441 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 857 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.258.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 289, total running count: 757 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.259.718 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.262.377 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 858 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.274.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 290, total running count: 758 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.275.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.277.887 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 859 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.289.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 291, total running count: 759 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.290.740 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.292.409 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 860 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:39.305.459 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 292, total running count: 760 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.306.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.308.260 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 861 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.321.139 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 293, total running count: 761 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.321.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.324.039 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 862 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.336.543 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 294, total running count: 762 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.337.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.339.711 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 863 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.352.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 295, total running count: 763 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.352.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.355.393 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 864 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.367.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 296, total running count: 764 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.368.378 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.370.975 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 865 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.383.018 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 297, total running count: 765 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.383.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.385.662 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 866 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:39.398.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 298, total running count: 766 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.399.173 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.401.095 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 867 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.413.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 299, total running count: 767 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:39.414.887 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.416.533 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 868 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.429.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 300, total running count: 768 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.430.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.432.213 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 869 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.444.998 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 301, total running count: 769 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.446.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.447.868 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 870 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.460.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 302, total running count: 770 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.461.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.463.799 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 871 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.476.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 303, total running count: 771 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.477.000 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.479.510 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 872 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.491.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 304, total running count: 772 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.492.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.495.250 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 873 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.507.347 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 305, total running count: 773 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.508.173 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.509.910 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 874 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.522.887 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 306, total running count: 774 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.523.637 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.525.746 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 875 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.538.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 307, total running count: 775 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.539.259 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.541.302 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 876 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:39.553.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 308, total running count: 776 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.554.865 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.556.834 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 877 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.569.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 309, total running count: 777 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.570.382 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.572.456 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 878 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.585.028 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 310, total running count: 778 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.585.781 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.588.234 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 879 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.600.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 311, total running count: 779 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.601.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.603.853 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 880 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.616.192 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 312, total running count: 780 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.616.861 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.619.202 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 881 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.631.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 313, total running count: 781 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.632.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.634.755 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 882 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.647.162 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 314, total running count: 782 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.648.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.650.334 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 883 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.662.757 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 315, total running count: 783 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.663.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.665.776 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 884 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.678.392 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 316, total running count: 784 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:39.679.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.681.407 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 885 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.693.951 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 317, total running count: 785 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.694.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.696.984 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 886 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.709.424 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 318, total running count: 786 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.710.255 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.712.915 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 887 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.724.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 319, total running count: 787 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.725.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.727.401 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 888 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.740.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 320, total running count: 788 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.741.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.742.769 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 889 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.755.686 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 321, total running count: 789 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.756.522 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.758.235 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 890 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.771.186 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 322, total running count: 790 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.772.045 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.773.562 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 891 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.786.647 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 323, total running count: 791 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.787.400 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.789.048 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 892 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.802.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 324, total running count: 792 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.802.992 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.804.573 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 893 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:39.817.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 325, total running count: 793 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.818.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.821.126 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 894 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.833.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 326, total running count: 794 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.834.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.837.058 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 895 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.849.281 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 327, total running count: 795 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.850.114 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.852.118 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 896 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.864.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 328, total running count: 796 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.865.865 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.868.111 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 897 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:39.880.776 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 329, total running count: 797 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:39.881.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.883.919 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 898 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.896.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 330, total running count: 798 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:39.897.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.899.748 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 899 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.912.370 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 331, total running count: 799 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:39.913.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.915.514 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 900 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:39.928.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 332, total running count: 800 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.929.039 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.931.374 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 901 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.944.079 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 333, total running count: 801 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.944.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.947.302 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 902 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:39.959.541 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 334, total running count: 802 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.960.485 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.963.152 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 903 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:39.975.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 335, total running count: 803 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.976.287 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.978.868 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 904 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:39.991.378 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 336, total running count: 804 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:39.992.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:39.994.803 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 905 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.007.051 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 337, total running count: 805 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.008.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.009.822 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 906 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.023.162 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 338, total running count: 806 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.023.716 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.025.808 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 907 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:40.038.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 339, total running count: 807 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.039.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.041.812 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 908 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:40.054.694 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 340, total running count: 808 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.055.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.057.814 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 909 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.070.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 341, total running count: 809 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.070.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.072.592 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 910 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.086.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 342, total running count: 810 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.086.703 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.088.888 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 911 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.101.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 343, total running count: 811 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.102.465 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.104.952 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 912 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.117.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 344, total running count: 812 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.118.235 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.119.858 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 913 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:40.133.007 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 345, total running count: 813 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.134.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.136.700 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 914 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.149.348 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 346, total running count: 814 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.150.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.152.434 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 915 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.165.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 347, total running count: 815 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.166.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.168.129 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 916 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.181.091 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 348, total running count: 816 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:40.181.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.183.824 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 917 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.196.660 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 349, total running count: 817 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:40.197.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.199.485 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 918 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:40.212.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 350, total running count: 818 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.212.848 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.215.346 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 919 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.227.686 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 351, total running count: 819 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:40.228.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.230.845 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 920 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.243.129 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 352, total running count: 820 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.244.052 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.246.509 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 921 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:40.258.946 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 353, total running count: 821 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:40.259.719 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.262.186 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 922 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:40.274.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 354, total running count: 822 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.275.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.278.103 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 923 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.290.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 355, total running count: 823 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.290.900 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.292.568 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 924 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.305.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 356, total running count: 824 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:40.306.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.309.149 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 925 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.321.112 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 357, total running count: 825 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:40.322.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.323.891 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 926 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.336.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 358, total running count: 826 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.337.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.339.826 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 927 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:40.352.400 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 359, total running count: 827 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.353.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.355.386 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 928 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.368.077 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 360, total running count: 828 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:40.369.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.370.889 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 929 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.383.811 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 361, total running count: 829 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:40.384.484 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.386.566 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 930 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:40.399.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 362, total running count: 830 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.400.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.402.330 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 931 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.414.858 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 363, total running count: 831 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:40.415.443 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.418.045 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 932 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.430.122 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 364, total running count: 832 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.430.930 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.432.770 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 933 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.445.815 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 365, total running count: 833 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.446.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.448.526 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 934 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.461.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 366, total running count: 834 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.462.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.463.979 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 935 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.476.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 367, total running count: 835 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.477.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.479.545 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 936 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.492.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 368, total running count: 836 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.492.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.495.108 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 937 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.507.584 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 369, total running count: 837 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.508.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.510.796 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 938 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.523.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 370, total running count: 838 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.524.114 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.526.316 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 939 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:40.538.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 371, total running count: 839 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:40.539.647 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.541.825 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 940 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.554.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 372, total running count: 840 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.555.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.557.391 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 941 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:40.569.928 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 373, total running count: 841 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.570.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.573.329 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 942 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.585.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 374, total running count: 842 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:40.586.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.588.171 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 943 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.601.338 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 375, total running count: 843 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.602.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.604.099 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 944 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:40.616.949 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 376, total running count: 844 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.617.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.619.844 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 945 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:40.632.683 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 377, total running count: 845 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.633.285 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.635.791 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 946 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.647.930 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 378, total running count: 846 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:40.649.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.651.797 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 947 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.663.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 379, total running count: 847 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:40.664.935 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.667.603 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 948 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.679.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 380, total running count: 848 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.680.381 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.682.218 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 949 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:40.695.136 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 381, total running count: 849 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.695.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.697.980 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 950 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:40.710.788 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 382, total running count: 850 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.711.813 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.713.792 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 951 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:40.726.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 383, total running count: 851 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.727.551 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.729.646 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 952 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:40.742.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 384, total running count: 852 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:40.743.260 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.745.586 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 953 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.758.071 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 385, total running count: 853 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:40.758.718 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.761.493 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 954 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.773.606 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 386, total running count: 854 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:40.774.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.776.225 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 955 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.789.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 387, total running count: 855 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.789.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.791.970 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 956 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.804.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 388, total running count: 856 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.805.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.807.646 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 957 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.820.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 389, total running count: 857 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.821.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.823.556 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 958 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.835.727 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 390, total running count: 858 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.836.926 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.839.502 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 959 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.851.732 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 391, total running count: 859 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.852.515 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.854.108 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 960 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.867.238 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 392, total running count: 860 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.868.009 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.869.801 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 961 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.882.675 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 393, total running count: 861 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.883.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.885.227 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 962 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.898.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 394, total running count: 862 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.899.244 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.900.827 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 963 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.913.922 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 395, total running count: 863 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.914.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.916.580 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 964 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.929.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 396, total running count: 864 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.930.239 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.932.155 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 965 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:40.944.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 397, total running count: 865 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:40.945.905 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.947.705 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 966 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:40.960.613 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 398, total running count: 866 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.961.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.963.502 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 967 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.976.263 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 399, total running count: 867 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:40.977.099 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.979.162 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 968 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:40.991.876 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 400, total running count: 868 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:40.992.693 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:40.994.743 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 969 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.007.349 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 401, total running count: 869 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:41.008.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.010.659 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 970 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.022.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 402, total running count: 870 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:41.023.793 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.026.459 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 971 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.038.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 403, total running count: 871 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.039.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.041.854 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 972 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.054.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 404, total running count: 872 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.054.960 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.056.925 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 973 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.069.856 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 405, total running count: 873 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.070.641 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.072.670 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 974 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.085.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 406, total running count: 874 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.086.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.088.386 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 975 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:41.101.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 407, total running count: 875 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.101.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.103.951 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 976 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.116.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 408, total running count: 876 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.117.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.119.484 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 977 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.132.145 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 409, total running count: 877 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.133.106 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.134.954 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 978 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.147.948 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 410, total running count: 878 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:41.148.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.150.822 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 979 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.163.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 411, total running count: 879 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:41.163.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.166.682 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 980 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.178.661 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 412, total running count: 880 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.179.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.181.416 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 981 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:41.194.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 413, total running count: 881 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.194.943 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.197.422 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 982 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.209.815 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 414, total running count: 882 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.210.447 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.213.270 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 983 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.225.260 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 415, total running count: 883 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.225.947 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.227.739 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 984 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.240.771 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 416, total running count: 884 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.241.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.243.242 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 985 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.256.150 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 417, total running count: 885 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.257.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.258.833 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 986 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.271.926 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 418, total running count: 886 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.272.735 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.274.373 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 987 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.287.402 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 419, total running count: 887 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.288.313 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.290.176 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 988 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.303.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 420, total running count: 888 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.303.882 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.305.782 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 989 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.318.681 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 421, total running count: 889 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.319.392 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.321.571 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 990 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.334.330 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 422, total running count: 890 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.335.387 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.337.529 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 991 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.350.325 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 423, total running count: 891 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.350.909 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.353.333 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 992 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.365.484 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 424, total running count: 892 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:41.366.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.368.054 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 993 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.381.187 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 425, total running count: 893 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.381.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.383.924 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 994 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.396.693 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 426, total running count: 894 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.397.730 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.399.629 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 995 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.412.611 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 427, total running count: 895 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.413.178 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.415.304 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 996 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.427.939 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 428, total running count: 896 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.428.696 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.430.913 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 997 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.443.535 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 429, total running count: 897 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.444.077 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.445.785 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 998 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.458.928 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 430, total running count: 898 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.459.739 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.461.448 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 999 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.474.471 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 431, total running count: 899 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.475.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.477.359 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1000 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:41.489.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 432, total running count: 900 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.490.592 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.493.320 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1001 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.505.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 433, total running count: 901 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.506.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.508.117 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1002 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:41.520.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 434, total running count: 902 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.521.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.524.062 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1003 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.536.403 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 435, total running count: 903 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.537.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.539.719 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1004 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.552.212 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 436, total running count: 904 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.553.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.555.383 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1005 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.567.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 437, total running count: 905 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.568.637 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.571.099 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1006 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.583.425 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 438, total running count: 906 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.584.330 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.586.651 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1007 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.598.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 439, total running count: 907 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:41.599.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.602.133 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1008 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.614.660 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 440, total running count: 908 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:41.615.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.617.678 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1009 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.630.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 441, total running count: 909 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.630.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.633.340 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1010 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.645.480 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 442, total running count: 910 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.646.395 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.647.972 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1011 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.661.033 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 443, total running count: 911 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.662.044 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.663.920 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1012 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.676.742 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 444, total running count: 912 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.677.563 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.679.503 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1013 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.692.329 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 445, total running count: 913 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.693.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.695.421 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1014 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:41.708.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 446, total running count: 914 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:41.708.765 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.711.410 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1015 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.723.588 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 447, total running count: 915 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.724.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.726.176 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1016 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.739.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 448, total running count: 916 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.739.874 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.741.823 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1017 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.754.698 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 449, total running count: 917 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.755.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.757.662 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1018 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:41.770.344 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 450, total running count: 918 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.770.978 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.772.572 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1019 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.785.768 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 451, total running count: 919 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.786.514 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.788.259 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1020 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.801.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 452, total running count: 920 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.802.089 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.803.959 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1021 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.816.800 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 453, total running count: 921 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.817.822 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.819.792 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1022 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.832.668 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 454, total running count: 922 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.833.493 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.835.775 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1023 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.848.174 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 455, total running count: 923 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.849.389 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.851.522 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1024 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.864.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 456, total running count: 924 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.864.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.867.269 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1025 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.879.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 457, total running count: 925 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.880.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.882.111 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1026 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.895.317 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 458, total running count: 926 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.896.089 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.897.901 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1027 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:41.910.716 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 459, total running count: 927 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:41.911.608 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.913.837 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1028 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:41.926.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 460, total running count: 928 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:41.927.233 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.929.656 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1029 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.941.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 461, total running count: 929 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.942.888 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.945.490 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1030 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.957.795 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 462, total running count: 930 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.958.556 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.961.007 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1031 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.973.078 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 463, total running count: 931 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.974.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.975.764 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1032 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:41.988.650 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 464, total running count: 932 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:41.989.764 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:41.991.484 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1033 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.004.488 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 465, total running count: 933 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.005.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.007.278 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1034 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.020.037 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 466, total running count: 934 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.020.933 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.022.959 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1035 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.035.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 467, total running count: 935 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.036.433 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.038.577 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1036 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.051.218 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 468, total running count: 936 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:40:42.052.374 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_1 execution count: 2, execution time: 7278.79 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:40:42.052.517 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_1 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:40:42.052.614 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty epoch: 2 step: 468, loss is 2.307192325592041 [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:40:42.053.471 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.h:76] ContinueSend] continue send at the beginning of the epoch [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:40:42.054.592 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:40:42.054.654 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:40:42.054.696 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_1 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.054.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.056.735 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1037 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.069.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 1, total running count: 937 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.070.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.072.745 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1038 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.085.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 2, total running count: 938 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.086.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.089.722 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1039 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.101.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 3, total running count: 939 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.102.526 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.104.180 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1040 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.117.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 4, total running count: 940 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.118.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.119.810 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1041 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.132.962 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 5, total running count: 941 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.133.809 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.135.774 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1042 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.148.479 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 6, total running count: 942 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.149.412 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.151.670 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1043 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.163.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 7, total running count: 943 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.164.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.167.369 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1044 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.179.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 8, total running count: 944 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.180.389 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.182.010 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1045 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.195.187 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 9, total running count: 945 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.195.872 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.197.926 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1046 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.210.780 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 10, total running count: 946 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.211.412 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.213.791 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1047 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.226.138 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 11, total running count: 947 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.226.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.229.469 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1048 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.241.514 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 12, total running count: 948 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.242.663 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.244.293 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1049 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.257.514 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 13, total running count: 949 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.258.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.260.069 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1050 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.273.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 14, total running count: 950 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.274.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.276.672 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1051 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.288.839 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 15, total running count: 951 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.289.606 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.292.245 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1052 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.304.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 16, total running count: 952 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.305.106 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.306.804 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1053 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.319.630 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 17, total running count: 953 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.320.441 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.322.690 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1054 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.335.269 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 18, total running count: 954 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.335.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.338.348 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1055 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.350.544 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 19, total running count: 955 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.351.580 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.353.678 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1056 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.366.329 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 20, total running count: 956 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.367.013 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.369.229 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1057 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.381.484 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 21, total running count: 957 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.382.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.384.170 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1058 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.397.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 22, total running count: 958 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.397.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.399.863 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1059 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.412.640 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 23, total running count: 959 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.413.343 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.415.386 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1060 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.428.030 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 24, total running count: 960 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.429.045 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.430.872 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1061 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.443.731 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 25, total running count: 961 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.444.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.446.433 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1062 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.459.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 26, total running count: 962 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.460.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.462.247 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1063 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.475.059 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 27, total running count: 963 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.475.523 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.478.055 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1064 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.490.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 28, total running count: 964 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.491.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.492.670 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1065 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.505.677 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 29, total running count: 965 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.506.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.508.324 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1066 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.521.141 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 30, total running count: 966 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.522.134 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.524.111 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1067 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.536.675 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 31, total running count: 967 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.537.389 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.539.829 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1068 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.552.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 32, total running count: 968 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.552.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.555.319 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1069 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.567.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 33, total running count: 969 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.568.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.570.714 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1070 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.583.141 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 34, total running count: 970 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.583.783 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.586.245 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1071 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.598.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 35, total running count: 971 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.599.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.601.772 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1072 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.613.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 36, total running count: 972 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.614.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.617.357 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1073 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.629.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 37, total running count: 973 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.630.246 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.631.971 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1074 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.644.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 38, total running count: 974 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.645.812 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.647.880 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1075 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.660.511 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 39, total running count: 975 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.660.997 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.663.472 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1076 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.675.444 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 40, total running count: 976 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.676.700 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.678.911 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1077 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.691.380 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 41, total running count: 977 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.692.121 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.694.508 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1078 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.706.668 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 42, total running count: 978 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.707.647 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.710.025 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1079 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.722.308 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 43, total running count: 979 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.723.212 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.725.654 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1080 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.737.789 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 44, total running count: 980 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.738.740 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.740.422 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1081 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.753.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 45, total running count: 981 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.754.249 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.756.138 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1082 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.768.888 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 46, total running count: 982 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.769.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.772.048 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1083 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.784.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 47, total running count: 983 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.785.061 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.786.660 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1084 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.799.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 48, total running count: 984 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.800.636 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.802.171 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1085 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.815.372 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 49, total running count: 985 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.815.909 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.817.628 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1086 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.830.614 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 50, total running count: 986 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.831.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.832.986 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1087 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.845.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 51, total running count: 987 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.846.586 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.848.735 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1088 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.861.163 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 52, total running count: 988 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.862.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.864.334 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1089 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.877.177 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 53, total running count: 989 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.877.531 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.880.261 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1090 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.892.598 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 54, total running count: 990 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.892.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.894.566 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1091 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.907.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 55, total running count: 991 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.908.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.911.165 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1092 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.923.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 56, total running count: 992 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.924.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.925.545 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1093 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.938.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 57, total running count: 993 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:42.939.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.942.082 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1094 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.954.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 58, total running count: 994 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:42.955.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.957.634 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1095 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:42.970.147 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 59, total running count: 995 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:42.970.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.973.136 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1096 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.985.633 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 60, total running count: 996 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:42.986.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:42.988.631 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1097 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.001.031 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 61, total running count: 997 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.001.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.004.136 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1098 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.016.476 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 62, total running count: 998 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.017.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.019.634 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1099 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.032.440 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 63, total running count: 999 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.033.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.035.469 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1100 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.048.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 64, total running count: 1000 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.048.433 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.050.040 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1101 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.063.443 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 65, total running count: 1001 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.063.800 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.065.837 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1102 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.078.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 66, total running count: 1002 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:43.079.265 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.081.677 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1103 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.094.249 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 67, total running count: 1003 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.094.596 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.096.114 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1104 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.109.574 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 68, total running count: 1004 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.109.978 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.111.652 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1105 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.124.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 69, total running count: 1005 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.125.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.127.324 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1106 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:43.140.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 70, total running count: 1006 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.141.097 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.142.946 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1107 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:43.156.080 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 71, total running count: 1007 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.156.551 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.158.780 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1108 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:43.171.554 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 72, total running count: 1008 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:43.172.059 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.174.258 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1109 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.187.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 73, total running count: 1009 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.187.471 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.189.739 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1110 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.202.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 74, total running count: 1010 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.202.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.205.076 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1111 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.217.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 75, total running count: 1011 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.218.420 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.220.478 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1112 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:43.233.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 76, total running count: 1012 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:43.233.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.236.034 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1113 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.248.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 77, total running count: 1013 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:43.249.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.251.488 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1114 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.264.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 78, total running count: 1014 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.265.015 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.267.316 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1115 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.280.124 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 79, total running count: 1015 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.280.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.282.788 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1116 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.295.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 80, total running count: 1016 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:43.295.890 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.298.513 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1117 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.310.815 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 81, total running count: 1017 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.311.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.313.136 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1118 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.326.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 82, total running count: 1018 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.327.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.328.998 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1119 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.342.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 83, total running count: 1019 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.342.532 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.344.570 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1120 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.357.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 84, total running count: 1020 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.357.925 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.360.072 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1121 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.372.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 85, total running count: 1021 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:43.373.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.375.762 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1122 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.388.247 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 86, total running count: 1022 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.388.746 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.390.364 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1123 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.403.703 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 87, total running count: 1023 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.404.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.406.160 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1124 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.419.416 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 88, total running count: 1024 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.419.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.421.632 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1125 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.435.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 89, total running count: 1025 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.435.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.437.216 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1126 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.450.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 90, total running count: 1026 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.451.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.452.705 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1127 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.466.027 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 91, total running count: 1027 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.466.554 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.468.189 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1128 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.481.522 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 92, total running count: 1028 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.482.032 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.483.784 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1129 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.497.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 93, total running count: 1029 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.497.601 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.499.574 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1130 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.512.739 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 94, total running count: 1030 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.513.192 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.515.321 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1131 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.528.224 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 95, total running count: 1031 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.529.615 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.531.284 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1132 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.544.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 96, total running count: 1032 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.545.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.546.870 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1133 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:43.560.131 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 97, total running count: 1033 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.560.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.562.388 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1134 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.575.413 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 98, total running count: 1034 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:43.575.770 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.577.937 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1135 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.590.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 99, total running count: 1035 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.591.015 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.592.574 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1136 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.605.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 100, total running count: 1036 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.606.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.608.271 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1137 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:43.621.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 101, total running count: 1037 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.622.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.623.886 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1138 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.637.073 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 102, total running count: 1038 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.637.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.639.374 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1139 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.653.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 103, total running count: 1039 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.653.715 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.656.094 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1140 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.669.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 104, total running count: 1040 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.669.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.671.618 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1141 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.684.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 105, total running count: 1041 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.685.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.687.098 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1142 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.700.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 106, total running count: 1042 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.700.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.702.569 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1143 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.716.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 107, total running count: 1043 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:43.716.453 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.718.476 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1144 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.731.697 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 108, total running count: 1044 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.732.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.734.341 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1145 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.747.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 109, total running count: 1045 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.747.417 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.749.825 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1146 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.762.486 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 110, total running count: 1046 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.762.831 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.765.371 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1147 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.777.885 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 111, total running count: 1047 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:43.778.224 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.780.841 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1148 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.793.394 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 112, total running count: 1048 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.793.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.795.528 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1149 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.808.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 113, total running count: 1049 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.809.247 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.811.309 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1150 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.824.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 114, total running count: 1050 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:43.824.554 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.826.911 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1151 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:43.839.541 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 115, total running count: 1051 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.839.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.842.365 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1152 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.854.868 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 116, total running count: 1052 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.855.210 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.856.769 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1153 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.870.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 117, total running count: 1053 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.870.640 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.872.365 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1154 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.885.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 118, total running count: 1054 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.885.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.888.119 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1155 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.900.960 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 119, total running count: 1055 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.901.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.902.810 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1156 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.916.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 120, total running count: 1056 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.916.550 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.918.477 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1157 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.931.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 121, total running count: 1057 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.931.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.934.066 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1158 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.949.804 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 122, total running count: 1058 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:43.950.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.951.874 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1159 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.966.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 123, total running count: 1059 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:43.966.420 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.968.750 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1160 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:43.981.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 124, total running count: 1060 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.981.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.984.589 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1161 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:43.997.031 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 125, total running count: 1061 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:43.997.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:43.998.968 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1162 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.012.562 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 126, total running count: 1062 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.012.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.014.520 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1163 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:44.028.151 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 127, total running count: 1063 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.028.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.030.553 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1164 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.043.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 128, total running count: 1064 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.044.002 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.046.255 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1165 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.059.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 129, total running count: 1065 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.059.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.061.793 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1166 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:44.074.531 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 130, total running count: 1066 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:44.074.874 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.077.356 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1167 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.089.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 131, total running count: 1067 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.090.106 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.092.722 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1168 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:44.105.379 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 132, total running count: 1068 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.105.769 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.108.534 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1169 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.120.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 133, total running count: 1069 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.121.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.123.238 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1170 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.136.275 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 134, total running count: 1070 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.136.617 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.138.940 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1171 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.151.759 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 135, total running count: 1071 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.152.107 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.154.744 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1172 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.167.148 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 136, total running count: 1072 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.167.478 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.169.351 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1173 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.182.615 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 137, total running count: 1073 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.182.955 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.184.945 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1174 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.197.851 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 138, total running count: 1074 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.198.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.200.628 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1175 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.213.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 139, total running count: 1075 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.213.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.215.323 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1176 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.228.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 140, total running count: 1076 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.228.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.230.928 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1177 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.244.092 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 141, total running count: 1077 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.244.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.246.558 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1178 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.260.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 142, total running count: 1078 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.260.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.262.703 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1179 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.275.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 143, total running count: 1079 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.276.260 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.278.323 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1180 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.291.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 144, total running count: 1080 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.291.818 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.294.064 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1181 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.307.099 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 145, total running count: 1081 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.307.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.310.076 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1182 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:44.322.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 146, total running count: 1082 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.322.843 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.325.649 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1183 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:44.337.897 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 147, total running count: 1083 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.338.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.339.881 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1184 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.353.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 148, total running count: 1084 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.353.660 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.355.479 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1185 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.368.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 149, total running count: 1085 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.369.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.371.277 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1186 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.384.214 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 150, total running count: 1086 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.384.554 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.387.154 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1187 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.399.645 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 151, total running count: 1087 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.399.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.401.602 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1188 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.416.879 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 152, total running count: 1088 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.417.236 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.419.705 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1189 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.432.874 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 153, total running count: 1089 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.433.226 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.435.725 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1190 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.448.599 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 154, total running count: 1090 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:44.448.945 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.451.451 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1191 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.464.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 155, total running count: 1091 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.464.612 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.466.943 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1192 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:44.479.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 156, total running count: 1092 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.480.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.482.325 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1193 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.495.616 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 157, total running count: 1093 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.495.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.497.681 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1194 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.511.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 158, total running count: 1094 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:44.511.396 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.513.285 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1195 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.526.515 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 159, total running count: 1095 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.526.859 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.528.884 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1196 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.541.949 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 160, total running count: 1096 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.542.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.544.742 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1197 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.561.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 161, total running count: 1097 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.561.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.564.126 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1198 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:44.577.178 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 162, total running count: 1098 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.577.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.579.746 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1199 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.592.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 163, total running count: 1099 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.593.065 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.595.316 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1200 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:44.608.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 164, total running count: 1100 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:44.608.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.611.082 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1201 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.623.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 165, total running count: 1101 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.624.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.625.730 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1202 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.639.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 166, total running count: 1102 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.639.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.641.187 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1203 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.654.806 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 167, total running count: 1103 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:44.655.148 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.656.695 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1204 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:44.670.213 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 168, total running count: 1104 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:44.670.545 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.672.182 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1205 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.685.629 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 169, total running count: 1105 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.686.011 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.687.716 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1206 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.700.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 170, total running count: 1106 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.701.238 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.703.493 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1207 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.716.799 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 171, total running count: 1107 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.719.247 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.721.748 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1208 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.734.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 172, total running count: 1108 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.734.843 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.737.351 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1209 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.750.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 173, total running count: 1109 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.750.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.752.953 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1210 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.765.583 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 174, total running count: 1110 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.766.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.767.739 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1211 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.781.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 175, total running count: 1111 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.781.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.783.368 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1212 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:44.796.744 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 176, total running count: 1112 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.797.097 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.798.953 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1213 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.812.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 177, total running count: 1113 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.812.681 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.814.483 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1214 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.827.736 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 178, total running count: 1114 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.829.425 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.831.418 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1215 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.844.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 179, total running count: 1115 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.845.943 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.848.470 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1216 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.861.208 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 180, total running count: 1116 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.861.806 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.864.115 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1217 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.877.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 181, total running count: 1117 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.877.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.879.461 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1218 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:44.892.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 182, total running count: 1118 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.893.329 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.894.883 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1219 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.908.370 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 183, total running count: 1119 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.908.890 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.911.339 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1220 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:44.924.058 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 184, total running count: 1120 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.926.089 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.927.943 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1221 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.941.287 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 185, total running count: 1121 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.944.244 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.945.909 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1222 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:44.959.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 186, total running count: 1122 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.959.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.961.717 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1223 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.975.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 187, total running count: 1123 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:44.975.569 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.977.490 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1224 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:44.990.713 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 188, total running count: 1124 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:44.991.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:44.993.194 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1225 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.006.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 189, total running count: 1125 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.007.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.008.945 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1226 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.022.071 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 190, total running count: 1126 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:45.022.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.024.887 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1227 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.037.586 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 191, total running count: 1127 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.040.205 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.042.776 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1228 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.055.289 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 192, total running count: 1128 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.055.789 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.058.200 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1229 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.070.998 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 193, total running count: 1129 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.071.508 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.073.578 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1230 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.086.516 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 194, total running count: 1130 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.086.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.089.337 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1231 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.101.874 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 195, total running count: 1131 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.102.238 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.103.891 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1232 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.117.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 196, total running count: 1132 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.117.897 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.119.806 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1233 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.132.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 197, total running count: 1133 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:45.135.498 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.137.778 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1234 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.150.620 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 198, total running count: 1134 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.152.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.154.707 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1235 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.167.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 199, total running count: 1135 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.168.100 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.170.535 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1236 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.183.106 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 200, total running count: 1136 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.184.142 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.186.434 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1237 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.199.232 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 201, total running count: 1137 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.199.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.201.890 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1238 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:45.214.839 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 202, total running count: 1138 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.215.749 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.217.790 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1239 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.230.800 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 203, total running count: 1139 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.231.394 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.233.758 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1240 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.246.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 204, total running count: 1140 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:45.253.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.255.107 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1241 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.268.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 205, total running count: 1141 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.271.086 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.273.235 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1242 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.286.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 206, total running count: 1142 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.287.054 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.289.008 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1243 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:45.302.066 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 207, total running count: 1143 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:45.302.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.304.565 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1244 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.317.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 208, total running count: 1144 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.318.961 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.321.213 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1245 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.334.105 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 209, total running count: 1145 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.334.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.336.983 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1246 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:45.349.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 210, total running count: 1146 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.351.358 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.353.959 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1247 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.366.439 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 211, total running count: 1147 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.367.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.369.763 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1248 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.382.141 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 212, total running count: 1148 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.383.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.385.423 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1249 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.398.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 213, total running count: 1149 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.398.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.400.844 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1250 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.413.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 214, total running count: 1150 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.414.582 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.416.541 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1251 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.429.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 215, total running count: 1151 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.430.313 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.432.409 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1252 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.445.349 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 216, total running count: 1152 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.446.295 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.448.071 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1253 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:45.461.417 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 217, total running count: 1153 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.462.707 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.464.735 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1254 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:45.477.841 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 218, total running count: 1154 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.478.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.480.385 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1255 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.493.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 219, total running count: 1155 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.494.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.495.844 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1256 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.509.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 220, total running count: 1156 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.509.758 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.511.517 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1257 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.524.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 221, total running count: 1157 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.525.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.527.372 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1258 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.540.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 222, total running count: 1158 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:45.541.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.544.023 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1259 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.556.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 223, total running count: 1159 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:45.557.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.559.523 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1260 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.572.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 224, total running count: 1160 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.573.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.575.254 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1261 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.588.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 225, total running count: 1161 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.588.926 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.591.062 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1262 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.604.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 226, total running count: 1162 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.604.647 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.606.876 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1263 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.619.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 227, total running count: 1163 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.620.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.622.390 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1264 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.635.380 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 228, total running count: 1164 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.635.775 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.637.863 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1265 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.650.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 229, total running count: 1165 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.651.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.653.356 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1266 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.666.453 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 230, total running count: 1166 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.666.946 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.669.049 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1267 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:45.682.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 231, total running count: 1167 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.682.647 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.684.922 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1268 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.697.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 232, total running count: 1168 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:45.698.362 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.700.565 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1269 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.713.460 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 233, total running count: 1169 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:45.714.138 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.716.429 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1270 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:45.729.032 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 234, total running count: 1170 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:45.729.708 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.732.161 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1271 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.744.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 235, total running count: 1171 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.745.317 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.747.760 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1272 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.760.311 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 236, total running count: 1172 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.760.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.763.316 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1273 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.776.208 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 237, total running count: 1173 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.776.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.778.748 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1274 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.791.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 238, total running count: 1174 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.792.187 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.794.162 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1275 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.807.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 239, total running count: 1175 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.807.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.809.906 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1276 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.822.640 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 240, total running count: 1176 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:45.823.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.824.812 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1277 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.838.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 241, total running count: 1177 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.838.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.840.445 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1278 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.853.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 242, total running count: 1178 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.854.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.856.076 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1279 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.869.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 243, total running count: 1179 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.870.053 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.871.784 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1280 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.885.102 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 244, total running count: 1180 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.885.447 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.887.500 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1281 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.900.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 245, total running count: 1181 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:45.900.888 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.902.862 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1282 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:45.915.898 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 246, total running count: 1182 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.916.525 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.918.502 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1283 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.931.558 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 247, total running count: 1183 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.931.938 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.934.023 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1284 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.946.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 248, total running count: 1184 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:45.947.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.949.713 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1285 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.962.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 249, total running count: 1185 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:45.963.173 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.965.564 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1286 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:45.978.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 250, total running count: 1186 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.978.663 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.981.261 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1287 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:45.993.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 251, total running count: 1187 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:45.994.246 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:45.996.926 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1288 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.009.155 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 252, total running count: 1188 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:46.009.909 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.012.539 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1289 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.025.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 253, total running count: 1189 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.025.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.028.214 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1290 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:46.040.596 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 254, total running count: 1190 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.041.020 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.042.917 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1291 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.056.165 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 255, total running count: 1191 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.056.525 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.058.470 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1292 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.071.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 256, total running count: 1192 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.071.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.074.024 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1293 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.087.151 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 257, total running count: 1193 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.087.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.089.569 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1294 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.102.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 258, total running count: 1194 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.103.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.105.604 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1295 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.118.404 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 259, total running count: 1195 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.119.165 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.121.366 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1296 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.133.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 260, total running count: 1196 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.134.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.137.432 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1297 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.149.791 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 261, total running count: 1197 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:46.150.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.151.929 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1298 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:46.165.247 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 262, total running count: 1198 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.165.587 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.167.738 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1299 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.180.705 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 263, total running count: 1199 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.181.059 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.183.624 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1300 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.196.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 264, total running count: 1200 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:46.196.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.199.338 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1301 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.211.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 265, total running count: 1201 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.212.139 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.213.991 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1302 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.227.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 266, total running count: 1202 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.227.640 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.229.822 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1303 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.242.752 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 267, total running count: 1203 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:46.243.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.245.389 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1304 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.258.202 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 268, total running count: 1204 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:46.258.551 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.260.999 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1305 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.273.589 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 269, total running count: 1205 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.273.945 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.276.492 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1306 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.288.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 270, total running count: 1206 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:46.289.319 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.291.935 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1307 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.304.247 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 271, total running count: 1207 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.304.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.306.311 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1308 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.319.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 272, total running count: 1208 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.320.032 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.321.795 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1309 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.335.154 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 273, total running count: 1209 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.335.793 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.338.450 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1310 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.350.777 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 274, total running count: 1210 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:46.351.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.354.070 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1311 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.366.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 275, total running count: 1211 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.366.970 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.368.597 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1312 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.381.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 276, total running count: 1212 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:46.382.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.384.186 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1313 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.397.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 277, total running count: 1213 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.398.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.399.813 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1314 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.412.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 278, total running count: 1214 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.413.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.415.424 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1315 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.428.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 279, total running count: 1215 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.428.789 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.430.919 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1316 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.443.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 280, total running count: 1216 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.444.204 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.446.259 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1317 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.459.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 281, total running count: 1217 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.459.890 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.461.816 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1318 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.474.928 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 282, total running count: 1218 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.475.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.477.428 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1319 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:46.490.482 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 283, total running count: 1219 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.490.848 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.493.274 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1320 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:46.505.816 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 284, total running count: 1220 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:46.506.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.508.948 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1321 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.521.677 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 285, total running count: 1221 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:46.522.278 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.524.474 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1322 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.537.252 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 286, total running count: 1222 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:46.537.890 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.540.080 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1323 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.552.844 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 287, total running count: 1223 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.553.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.555.841 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1324 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.568.252 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 288, total running count: 1224 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.568.611 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.570.488 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1325 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:46.583.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 289, total running count: 1225 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.584.183 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.586.234 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1326 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.599.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 290, total running count: 1226 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.600.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.602.113 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1327 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.615.142 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 291, total running count: 1227 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.615.551 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.617.473 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1328 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.630.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 292, total running count: 1228 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:46.630.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.633.016 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1329 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.646.107 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 293, total running count: 1229 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.646.474 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.648.590 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1330 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.661.539 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 294, total running count: 1230 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.661.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.664.493 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1331 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.677.015 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 295, total running count: 1231 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.677.374 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.678.915 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1332 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:46.692.324 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 296, total running count: 1232 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.692.889 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.694.527 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1333 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.707.960 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 297, total running count: 1233 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.708.442 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.710.107 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1334 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.723.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 298, total running count: 1234 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.723.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.725.755 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1335 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.739.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 299, total running count: 1235 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.739.353 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.741.380 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1336 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.754.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 300, total running count: 1236 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.755.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.757.311 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1337 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.770.424 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 301, total running count: 1237 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.771.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.773.944 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1338 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.786.345 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 302, total running count: 1238 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.786.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.789.364 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1339 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.801.738 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 303, total running count: 1239 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.802.097 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.804.063 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1340 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.816.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 304, total running count: 1240 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.817.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.819.892 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1341 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.832.590 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 305, total running count: 1241 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.832.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.834.515 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1342 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.847.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 306, total running count: 1242 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.848.361 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.850.197 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1343 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.863.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 307, total running count: 1243 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.864.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.866.201 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1344 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.879.216 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 308, total running count: 1244 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.879.574 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.881.806 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1345 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.894.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 309, total running count: 1245 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.895.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.897.242 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1346 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.910.289 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 310, total running count: 1246 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.910.738 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.912.655 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1347 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:46.925.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 311, total running count: 1247 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.926.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.928.046 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1348 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.941.395 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 312, total running count: 1248 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.942.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.943.707 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1349 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.957.066 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 313, total running count: 1249 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.957.530 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.959.414 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1350 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:46.972.612 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 314, total running count: 1250 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:46.973.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.974.956 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1351 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:46.988.264 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 315, total running count: 1251 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:46.988.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:46.990.512 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1352 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.003.775 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 316, total running count: 1252 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.004.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.006.041 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1353 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.019.441 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 317, total running count: 1253 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.020.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.021.802 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1354 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.035.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 318, total running count: 1254 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.035.722 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.038.348 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1355 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.050.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 319, total running count: 1255 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.051.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.052.642 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1356 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.066.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 320, total running count: 1256 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.066.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.069.249 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1357 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.082.096 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 321, total running count: 1257 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.082.460 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.085.088 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1358 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.097.598 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 322, total running count: 1258 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.098.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.099.780 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1359 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.113.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 323, total running count: 1259 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.113.756 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.116.333 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1360 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.128.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 324, total running count: 1260 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.129.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.131.901 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1361 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.144.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 325, total running count: 1261 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.145.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.147.307 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1362 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.160.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 326, total running count: 1262 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.160.872 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.163.130 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1363 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.176.289 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 327, total running count: 1263 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.176.740 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.178.854 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1364 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.191.987 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 328, total running count: 1264 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.192.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.194.745 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1365 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.207.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 329, total running count: 1265 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.209.778 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.212.157 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1366 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.225.239 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 330, total running count: 1266 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.225.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.228.197 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1367 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.241.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 331, total running count: 1267 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.241.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.244.213 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1368 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.256.997 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 332, total running count: 1268 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.257.381 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.259.994 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1369 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.272.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 333, total running count: 1269 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.273.421 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.276.044 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1370 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.288.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 334, total running count: 1270 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.289.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.291.911 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1371 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.304.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 335, total running count: 1271 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.305.128 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.307.769 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1372 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.320.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 336, total running count: 1272 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.320.805 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.322.829 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1373 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.336.358 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 337, total running count: 1273 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.336.738 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.338.766 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1374 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.351.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 338, total running count: 1274 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.352.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.354.437 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1375 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.367.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 339, total running count: 1275 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.368.171 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.370.601 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1376 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.383.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 340, total running count: 1276 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.383.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.386.693 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1377 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.399.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 341, total running count: 1277 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.399.657 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.402.305 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1378 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.415.007 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 342, total running count: 1278 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.415.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.417.135 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1379 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.430.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 343, total running count: 1279 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.430.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.433.004 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1380 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.446.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 344, total running count: 1280 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.446.675 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.448.963 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1381 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.461.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 345, total running count: 1281 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.462.354 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.464.914 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1382 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.477.657 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 346, total running count: 1282 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.478.068 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.480.293 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1383 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.493.248 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 347, total running count: 1283 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.493.625 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.496.137 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1384 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.509.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 348, total running count: 1284 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.509.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.511.769 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1385 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.524.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 349, total running count: 1285 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.525.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.527.473 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1386 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.540.701 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 350, total running count: 1286 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.542.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.544.360 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1387 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.557.744 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 351, total running count: 1287 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.558.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.560.075 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1388 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.573.531 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 352, total running count: 1288 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.573.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.575.620 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1389 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.589.138 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 353, total running count: 1289 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.589.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.591.201 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1390 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.604.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 354, total running count: 1290 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.604.969 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.606.946 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1391 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.620.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 355, total running count: 1291 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.620.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.622.740 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1392 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.635.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 356, total running count: 1292 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.636.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.638.808 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1393 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.651.265 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 357, total running count: 1293 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.651.612 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.653.318 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1394 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.666.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 358, total running count: 1294 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.667.665 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.670.065 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1395 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.682.800 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 359, total running count: 1295 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.683.705 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.685.642 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1396 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.698.663 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 360, total running count: 1296 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.699.379 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.701.193 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1397 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.714.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 361, total running count: 1297 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.714.872 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.716.858 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1398 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.730.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 362, total running count: 1298 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.730.444 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.732.442 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1399 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.745.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 363, total running count: 1299 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.746.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.747.918 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1400 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.761.311 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 364, total running count: 1300 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.761.850 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.763.543 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1401 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.776.889 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 365, total running count: 1301 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.777.239 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.779.354 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1402 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.792.382 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 366, total running count: 1302 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.792.737 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.795.121 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1403 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.807.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 367, total running count: 1303 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.808.202 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.810.830 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1404 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.823.424 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 368, total running count: 1304 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.823.780 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.826.547 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1405 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.838.909 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 369, total running count: 1305 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.839.291 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.841.086 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1406 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:47.854.517 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 370, total running count: 1306 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.854.882 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.856.494 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1407 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.869.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 371, total running count: 1307 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.870.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.872.250 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1408 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.885.800 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 372, total running count: 1308 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.886.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.888.950 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1409 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.901.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 373, total running count: 1309 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.902.106 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.904.741 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1410 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.917.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 374, total running count: 1310 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.917.618 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.919.284 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1411 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.932.945 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 375, total running count: 1311 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:47.933.319 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.935.260 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1412 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.948.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 376, total running count: 1312 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.948.897 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.950.877 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1413 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.964.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 377, total running count: 1313 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:47.964.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.966.524 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1414 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.979.717 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 378, total running count: 1314 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.980.114 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.982.238 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1415 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:47.995.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 379, total running count: 1315 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:47.995.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:47.998.180 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1416 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.010.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 380, total running count: 1316 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.011.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.013.840 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1417 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.026.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 381, total running count: 1317 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.029.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.031.602 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1418 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.044.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 382, total running count: 1318 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.046.244 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.048.462 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1419 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:48.061.192 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 383, total running count: 1319 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:48.061.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.063.978 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1420 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.077.087 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 384, total running count: 1320 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.077.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.079.478 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1421 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.092.521 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 385, total running count: 1321 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.092.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.095.246 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1422 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.107.961 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 386, total running count: 1322 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:48.108.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.111.069 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1423 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.123.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 387, total running count: 1323 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.123.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.125.562 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1424 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:48.139.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 388, total running count: 1324 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:48.139.370 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.141.531 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1425 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:48.154.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 389, total running count: 1325 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.155.084 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.157.102 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1426 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.170.349 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 390, total running count: 1326 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.170.903 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.172.852 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1427 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.186.016 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 391, total running count: 1327 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.186.380 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.188.496 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1428 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.201.439 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 392, total running count: 1328 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.202.132 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.204.016 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1429 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:48.217.155 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 393, total running count: 1329 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.217.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.219.703 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1430 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:48.232.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 394, total running count: 1330 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.233.330 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.235.650 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1431 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.248.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 395, total running count: 1331 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:48.249.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.251.520 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1432 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.264.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 396, total running count: 1332 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:48.264.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.267.346 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1433 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.280.134 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 397, total running count: 1333 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.280.570 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.283.104 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1434 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.295.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 398, total running count: 1334 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:48.296.056 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.297.807 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1435 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:48.311.182 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 399, total running count: 1335 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.311.658 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.313.773 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1436 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.326.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 400, total running count: 1336 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.327.291 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.329.357 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1437 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.342.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 401, total running count: 1337 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.342.926 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.345.282 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1438 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.358.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 402, total running count: 1338 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.358.742 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.361.341 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1439 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.373.789 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 403, total running count: 1339 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:48.374.460 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.376.089 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1440 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.389.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 404, total running count: 1340 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.390.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.393.056 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1441 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:48.405.511 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 405, total running count: 1341 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.405.876 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.407.761 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1442 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.420.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 406, total running count: 1342 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.421.249 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.423.491 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1443 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.436.287 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 407, total running count: 1343 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.436.820 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.439.335 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1444 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.452.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 408, total running count: 1344 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:48.452.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.455.070 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1445 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:48.467.489 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 409, total running count: 1345 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:48.467.851 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.469.791 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1446 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:48.482.939 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 410, total running count: 1346 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.483.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.485.378 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1447 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.498.713 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 411, total running count: 1347 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:48.499.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.501.227 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1448 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:48.514.271 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 412, total running count: 1348 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.514.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.517.157 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1449 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:48.529.614 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 413, total running count: 1349 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.530.263 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.532.864 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1450 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:48.545.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 414, total running count: 1350 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.545.840 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.548.432 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1451 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.560.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 415, total running count: 1351 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.561.353 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.563.024 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1452 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.576.609 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 416, total running count: 1352 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.577.142 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.578.848 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1453 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.592.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 417, total running count: 1353 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.592.690 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.594.776 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1454 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:48.607.897 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 418, total running count: 1354 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.608.263 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.610.395 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1455 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.623.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 419, total running count: 1355 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:48.623.840 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.626.213 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1456 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.639.008 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 420, total running count: 1356 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.639.582 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.642.209 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1457 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.654.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 421, total running count: 1357 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.655.127 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.656.711 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1458 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.670.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 422, total running count: 1358 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.670.597 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.672.399 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1459 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.685.717 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 423, total running count: 1359 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.686.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.688.310 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1460 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:48.701.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 424, total running count: 1360 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:48.701.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.704.319 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1461 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.716.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 425, total running count: 1361 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.717.329 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.718.961 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1462 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.732.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 426, total running count: 1362 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.732.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.734.564 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1463 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:48.747.930 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 427, total running count: 1363 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:48.748.331 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.750.201 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1464 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.763.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 428, total running count: 1364 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.764.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.765.728 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1465 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:48.779.054 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 429, total running count: 1365 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.779.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.781.438 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1466 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.794.750 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 430, total running count: 1366 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.795.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.798.438 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1467 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.810.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 431, total running count: 1367 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.811.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.813.233 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1468 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:48.826.622 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 432, total running count: 1368 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.827.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.828.948 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1469 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.842.182 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 433, total running count: 1369 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.842.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.844.656 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1470 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.857.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 434, total running count: 1370 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:48.858.158 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.860.179 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1471 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:48.873.178 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 435, total running count: 1371 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.873.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.875.697 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1472 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.888.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 436, total running count: 1372 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.889.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.891.460 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1473 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:48.904.106 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 437, total running count: 1373 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.904.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.907.388 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1474 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:48.920.045 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 438, total running count: 1374 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.920.479 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.922.979 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1475 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:48.935.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 439, total running count: 1375 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:48.935.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.937.596 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1476 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.951.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 440, total running count: 1376 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.951.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.953.786 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1477 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:48.966.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 441, total running count: 1377 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:48.967.158 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.969.373 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1478 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:48.982.244 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 442, total running count: 1378 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:48.982.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.984.966 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1479 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.997.765 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 443, total running count: 1379 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:48.998.127 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:48.999.785 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1480 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.013.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 444, total running count: 1380 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:49.013.668 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.015.699 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1481 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.028.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 445, total running count: 1381 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.029.140 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.031.238 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1482 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.044.033 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 446, total running count: 1382 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.044.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.046.909 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1483 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.059.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 447, total running count: 1383 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.059.680 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.061.409 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1484 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.074.928 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 448, total running count: 1384 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.075.277 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.077.060 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1485 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.090.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 449, total running count: 1385 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.090.641 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.092.924 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1486 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.105.782 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 450, total running count: 1386 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.106.138 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.108.680 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1487 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.121.134 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 451, total running count: 1387 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:49.121.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.123.347 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1488 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.136.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 452, total running count: 1388 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.137.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.139.215 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1489 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:49.152.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 453, total running count: 1389 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.152.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.154.799 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1490 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.167.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 454, total running count: 1390 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.167.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.170.224 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1491 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.182.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 455, total running count: 1391 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.183.352 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.185.814 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1492 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.198.483 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 456, total running count: 1392 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.198.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.200.414 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1493 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.213.722 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 457, total running count: 1393 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.214.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.216.260 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1494 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.229.130 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 458, total running count: 1394 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.229.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.232.160 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1495 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:49.244.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 459, total running count: 1395 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.245.118 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.246.836 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1496 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.260.358 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 460, total running count: 1396 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.261.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.263.680 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1497 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.276.625 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 461, total running count: 1397 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.276.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.279.508 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1498 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:49.292.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 462, total running count: 1398 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.292.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.294.348 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1499 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.307.408 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 463, total running count: 1399 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.307.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.309.891 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1500 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.322.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 464, total running count: 1400 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.323.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.325.628 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1501 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.338.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 465, total running count: 1401 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.338.908 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.341.385 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1502 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.353.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 466, total running count: 1402 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:49.354.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.356.060 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1503 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.369.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 467, total running count: 1403 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.369.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.371.796 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1504 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.385.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 468, total running count: 1404 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:40:49.385.652 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_1 execution count: 3, execution time: 7330.83 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:40:49.385.847 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_1 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:40:49.385.965 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty epoch: 3 step: 468, loss is 2.2931277751922607 [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:40:49.386.917 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.h:76] ContinueSend] continue send at the beginning of the epoch [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:40:49.388.062 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:40:49.388.125 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:40:49.388.168 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_1 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.388.530 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.391.128 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1505 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:49.403.798 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 1, total running count: 1405 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.404.644 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.406.913 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1506 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.419.889 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 2, total running count: 1406 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.420.302 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.422.430 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1507 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.435.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 3, total running count: 1407 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.436.039 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.438.118 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1508 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.451.200 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 4, total running count: 1408 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:49.451.808 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.453.589 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1509 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.466.993 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 5, total running count: 1409 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.467.358 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.469.557 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1510 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.482.348 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 6, total running count: 1410 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.482.793 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.485.207 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1511 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.497.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 7, total running count: 1411 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.498.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.500.902 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1512 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.513.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 8, total running count: 1412 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:49.514.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.516.690 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1513 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.529.543 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 9, total running count: 1413 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.530.224 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.532.741 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1514 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.545.308 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 10, total running count: 1414 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.545.709 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.547.358 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1515 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.560.705 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 11, total running count: 1415 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:49.561.442 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.564.080 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1516 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.576.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 12, total running count: 1416 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.577.007 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.579.717 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1517 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.592.149 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 13, total running count: 1417 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.592.522 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.594.380 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1518 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.607.575 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 14, total running count: 1418 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.607.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.610.296 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1519 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:49.622.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 15, total running count: 1419 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.623.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.626.073 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1520 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.638.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 16, total running count: 1420 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.638.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.640.447 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1521 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.653.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 17, total running count: 1421 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.654.407 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.656.251 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1522 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:49.669.381 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 18, total running count: 1422 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.669.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.671.937 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1523 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:49.684.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 19, total running count: 1423 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.685.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.687.544 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1524 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.700.226 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 20, total running count: 1424 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.700.883 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.702.946 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1525 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.715.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 21, total running count: 1425 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:49.716.550 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.718.451 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1526 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.731.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 22, total running count: 1426 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.731.975 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.734.043 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1527 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.746.841 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 23, total running count: 1427 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:49.747.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.749.585 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1528 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.762.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 24, total running count: 1428 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.762.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.765.267 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1529 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.777.755 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 25, total running count: 1429 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.778.343 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.780.980 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1530 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.793.427 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 26, total running count: 1430 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.793.845 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.795.392 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1531 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.808.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 27, total running count: 1431 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.809.371 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.811.054 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1532 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.824.531 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 28, total running count: 1432 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.824.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.826.593 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1533 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.839.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 29, total running count: 1433 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.840.252 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.842.497 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1534 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.855.344 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 30, total running count: 1434 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.855.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.858.259 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1535 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.870.663 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 31, total running count: 1435 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.871.008 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.872.621 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1536 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:49.886.122 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 32, total running count: 1436 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.886.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.888.100 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1537 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:49.901.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 33, total running count: 1437 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.901.845 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.903.788 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1538 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.916.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 34, total running count: 1438 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.917.265 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.919.620 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1539 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.932.346 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 35, total running count: 1439 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:49.932.687 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.935.287 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1540 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:49.947.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 36, total running count: 1440 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.948.270 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.950.714 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1541 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.963.244 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 37, total running count: 1441 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.963.575 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.966.247 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1542 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.978.560 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 38, total running count: 1442 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.978.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.980.913 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1543 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:49.993.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 39, total running count: 1443 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:49.994.304 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:49.996.611 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1544 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.009.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 40, total running count: 1444 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.009.872 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.012.347 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1545 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.024.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 41, total running count: 1445 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.025.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.026.750 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1546 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.040.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 42, total running count: 1446 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.040.485 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.042.294 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1547 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.055.441 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 43, total running count: 1447 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.055.798 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.057.816 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1548 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.070.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 44, total running count: 1448 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.071.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.073.301 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1549 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.086.218 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 45, total running count: 1449 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.086.769 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.088.944 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1550 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.101.951 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 46, total running count: 1450 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.102.313 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.104.630 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1551 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.117.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 47, total running count: 1451 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.117.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.119.310 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1552 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.132.609 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 48, total running count: 1452 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.133.000 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.134.830 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1553 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.147.912 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 49, total running count: 1453 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.148.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.150.471 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1554 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.163.484 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 50, total running count: 1454 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.163.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.165.895 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1555 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.179.039 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 51, total running count: 1455 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.179.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.181.555 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1556 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.194.653 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 52, total running count: 1456 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.195.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.197.279 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1557 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.210.072 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 53, total running count: 1457 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.210.695 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.213.040 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1558 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.225.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 54, total running count: 1458 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.226.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.228.551 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1559 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.240.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 55, total running count: 1459 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.241.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.244.055 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1560 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.256.488 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 56, total running count: 1460 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.256.853 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.259.452 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1561 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.271.936 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 57, total running count: 1461 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.272.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.274.019 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1562 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.287.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 58, total running count: 1462 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.287.750 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.289.348 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1563 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.302.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 59, total running count: 1463 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.303.173 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.304.858 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1564 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.318.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 60, total running count: 1464 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.318.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.320.400 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1565 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.333.827 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 61, total running count: 1465 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.334.394 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.335.939 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1566 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.349.379 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 62, total running count: 1466 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.349.805 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.351.577 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1567 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.364.812 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 63, total running count: 1467 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.365.264 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.367.454 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1568 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.380.484 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 64, total running count: 1468 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.380.890 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.383.344 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1569 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.395.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 65, total running count: 1469 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.396.394 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.398.905 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1570 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.411.358 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 66, total running count: 1470 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.411.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.414.320 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1571 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.426.828 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 67, total running count: 1471 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.427.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.429.685 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1572 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.442.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 68, total running count: 1472 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.442.748 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.445.340 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1573 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.457.638 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 69, total running count: 1473 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.458.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.459.719 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1574 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.472.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 70, total running count: 1474 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.473.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.475.618 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1575 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.488.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 71, total running count: 1475 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.488.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.491.218 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1576 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.503.794 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 72, total running count: 1476 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.504.289 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.506.818 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1577 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.519.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 73, total running count: 1477 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.519.731 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.522.169 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1578 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.534.758 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 74, total running count: 1478 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.535.132 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.537.591 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1579 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.550.033 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 75, total running count: 1479 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.552.800 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.555.295 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1580 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.567.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 76, total running count: 1480 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.568.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.570.638 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1581 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.583.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 77, total running count: 1481 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.583.562 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.586.197 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1582 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.598.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 78, total running count: 1482 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.598.936 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.601.487 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1583 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.613.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 79, total running count: 1483 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.614.164 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.615.906 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1584 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.629.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 80, total running count: 1484 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.629.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.631.326 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1585 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.644.614 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 81, total running count: 1485 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.645.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.646.823 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1586 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.660.140 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 82, total running count: 1486 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.660.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.663.488 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1587 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.676.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 83, total running count: 1487 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.676.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.678.972 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1588 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.691.572 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 84, total running count: 1488 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.692.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.694.460 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1589 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.707.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 85, total running count: 1489 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.707.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.709.814 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1590 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.722.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 86, total running count: 1490 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.723.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.725.238 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1591 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.738.354 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 87, total running count: 1491 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.738.840 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.740.890 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1592 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.753.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 88, total running count: 1492 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.754.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.756.546 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1593 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.769.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 89, total running count: 1493 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.769.828 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.772.376 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1594 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.784.695 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 90, total running count: 1494 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.785.304 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.787.899 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1595 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.800.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 91, total running count: 1495 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.800.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.803.422 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1596 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.816.138 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 92, total running count: 1496 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.816.878 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.818.830 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1597 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.831.916 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 93, total running count: 1497 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.832.347 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.834.290 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1598 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.847.381 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 94, total running count: 1498 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.847.889 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.849.761 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1599 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.862.960 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 95, total running count: 1499 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.863.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.865.268 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1600 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.878.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 96, total running count: 1500 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.879.106 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.881.740 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1601 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.894.222 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 97, total running count: 1501 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.894.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.897.288 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1602 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.909.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 98, total running count: 1502 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.910.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.911.833 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1603 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.924.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 99, total running count: 1503 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.925.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.927.744 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1604 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.940.382 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 100, total running count: 1504 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.940.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.943.251 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1605 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:50.955.945 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 101, total running count: 1505 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.956.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.958.754 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1606 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:50.971.535 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 102, total running count: 1506 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:50.971.890 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.974.282 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1607 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:50.986.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 103, total running count: 1507 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:50.987.372 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:50.989.795 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1608 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.002.460 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 104, total running count: 1508 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.002.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.005.646 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1609 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:51.018.001 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 105, total running count: 1509 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:51.018.462 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.020.195 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1610 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.033.544 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 106, total running count: 1510 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.034.058 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.035.948 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1611 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.049.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 107, total running count: 1511 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.049.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.051.698 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1612 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.064.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 108, total running count: 1512 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.065.598 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.067.303 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1613 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.080.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 109, total running count: 1513 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.081.020 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.082.899 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1614 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.096.077 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 110, total running count: 1514 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.096.459 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.098.678 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1615 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:51.111.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 111, total running count: 1515 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.112.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.114.529 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1616 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.127.358 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 112, total running count: 1516 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.127.739 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.130.090 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1617 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.142.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 113, total running count: 1517 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.143.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.145.761 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1618 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.158.267 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 114, total running count: 1518 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.158.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.161.266 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1619 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.173.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 115, total running count: 1519 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.174.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.176.925 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1620 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.189.262 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 116, total running count: 1520 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:51.189.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.191.497 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1621 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.204.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 117, total running count: 1521 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.205.265 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.207.291 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1622 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.220.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 118, total running count: 1522 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.220.683 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.222.809 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1623 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.235.697 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 119, total running count: 1523 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:51.236.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.238.447 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1624 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.251.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 120, total running count: 1524 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.251.796 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.254.080 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1625 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.266.975 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 121, total running count: 1525 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.267.637 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.270.181 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1626 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:51.282.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 122, total running count: 1526 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.282.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.284.560 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1627 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.298.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 123, total running count: 1527 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.298.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.301.465 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1628 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:51.313.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 124, total running count: 1528 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.314.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.316.004 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1629 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.329.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 125, total running count: 1529 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.329.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.332.029 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1630 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.344.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 126, total running count: 1530 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.345.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.347.501 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1631 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.360.300 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 127, total running count: 1531 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.360.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.362.982 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1632 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.375.567 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 128, total running count: 1532 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.375.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.378.405 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1633 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.390.969 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 129, total running count: 1533 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.391.302 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.393.802 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1634 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.406.514 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 130, total running count: 1534 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.406.876 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.409.290 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1635 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.421.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 131, total running count: 1535 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.422.264 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.424.653 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1636 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.437.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 132, total running count: 1536 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.437.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.440.291 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1637 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.452.739 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 133, total running count: 1537 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.453.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.455.812 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1638 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.468.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 134, total running count: 1538 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.468.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.471.276 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1639 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.484.046 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 135, total running count: 1539 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.484.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.486.810 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1640 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.499.592 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 136, total running count: 1540 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.500.269 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.502.229 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1641 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.515.331 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 137, total running count: 1541 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.515.695 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.517.834 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1642 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.530.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 138, total running count: 1542 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.531.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.533.550 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1643 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.546.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 139, total running count: 1543 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:51.546.738 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.549.279 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1644 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:51.561.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 140, total running count: 1544 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.562.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.565.050 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1645 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.577.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 141, total running count: 1545 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.577.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.580.531 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1646 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:51.592.831 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 142, total running count: 1546 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.593.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.594.856 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1647 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.608.387 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 143, total running count: 1547 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.608.766 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.610.553 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1648 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:51.623.758 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 144, total running count: 1548 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.624.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.626.273 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1649 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.639.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 145, total running count: 1549 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:51.639.675 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.642.252 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1650 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.654.724 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 146, total running count: 1550 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.655.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.656.839 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1651 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.670.111 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 147, total running count: 1551 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.670.552 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.672.550 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1652 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.685.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 148, total running count: 1552 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.685.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.688.381 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1653 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.700.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 149, total running count: 1553 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.701.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.703.984 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1654 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.716.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 150, total running count: 1554 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.716.850 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.719.476 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1655 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.732.045 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 151, total running count: 1555 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:51.732.474 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.734.022 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1656 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:51.747.479 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 152, total running count: 1556 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:51.747.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.749.893 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1657 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.762.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 153, total running count: 1557 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.763.291 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.765.536 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1658 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.778.318 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 154, total running count: 1558 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.778.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.781.035 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1659 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.793.898 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 155, total running count: 1559 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.794.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.796.583 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1660 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.809.427 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 156, total running count: 1560 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.809.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.812.128 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1661 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:51.824.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 157, total running count: 1561 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.825.128 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.827.662 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1662 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.840.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 158, total running count: 1562 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.840.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.842.484 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1663 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.855.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 159, total running count: 1563 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.856.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.858.169 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1664 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:51.871.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 160, total running count: 1564 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.871.543 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.873.759 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1665 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.886.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 161, total running count: 1565 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:51.887.036 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.889.454 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1666 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.902.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 162, total running count: 1566 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.902.601 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.905.249 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1667 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:51.917.609 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 163, total running count: 1567 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.918.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.919.771 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1668 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.933.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 164, total running count: 1568 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.933.459 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.935.409 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1669 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.948.587 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 165, total running count: 1569 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.948.955 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.950.806 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1670 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:51.964.096 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 166, total running count: 1570 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:51.964.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.966.274 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1671 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.979.602 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 167, total running count: 1571 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:51.979.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.982.031 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1672 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.995.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 168, total running count: 1572 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:51.995.348 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:51.997.788 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1673 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.010.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 169, total running count: 1573 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.010.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.012.539 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1674 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:52.025.683 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 170, total running count: 1574 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.026.042 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.027.977 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1675 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.041.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 171, total running count: 1575 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:52.041.601 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.043.395 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1676 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.056.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 172, total running count: 1576 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.056.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.058.810 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1677 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:52.071.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 173, total running count: 1577 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.072.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.074.147 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1678 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.087.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 174, total running count: 1578 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.087.609 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.089.660 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1679 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.102.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 175, total running count: 1579 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.103.135 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.105.155 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1680 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.118.186 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 176, total running count: 1580 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.118.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.120.884 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1681 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.133.486 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 177, total running count: 1581 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.133.878 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.135.674 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1682 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.149.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 178, total running count: 1582 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.149.730 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.152.297 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1683 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.165.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 179, total running count: 1583 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.165.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.167.849 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1684 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.180.344 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 180, total running count: 1584 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.180.694 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.183.256 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1685 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.195.856 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 181, total running count: 1585 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.196.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.198.748 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1686 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.211.108 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 182, total running count: 1586 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.211.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.213.034 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1687 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.226.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 183, total running count: 1587 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.226.858 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.228.504 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1688 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.241.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 184, total running count: 1588 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.242.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.243.849 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1689 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.257.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 185, total running count: 1589 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:52.257.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.259.365 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1690 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.272.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 186, total running count: 1590 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.273.100 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.274.848 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1691 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.288.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 187, total running count: 1591 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.288.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.290.543 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1692 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.303.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 188, total running count: 1592 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.303.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.306.493 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1693 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.318.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 189, total running count: 1593 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.319.251 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.321.218 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1694 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.334.353 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 190, total running count: 1594 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.334.771 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.337.052 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1695 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:52.351.407 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 191, total running count: 1595 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.351.795 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.354.061 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1696 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:52.367.109 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 192, total running count: 1596 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.367.496 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.369.758 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1697 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.382.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 193, total running count: 1597 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.383.121 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.385.328 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1698 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.398.395 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 194, total running count: 1598 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:52.398.756 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.401.077 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1699 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.414.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 195, total running count: 1599 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.414.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.416.761 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1700 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.429.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 196, total running count: 1600 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:52.430.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.432.325 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1701 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.445.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 197, total running count: 1601 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.445.533 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.447.878 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1702 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.460.737 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 198, total running count: 1602 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.461.091 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.463.407 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1703 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.476.425 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 199, total running count: 1603 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.476.771 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.479.049 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1704 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.491.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 200, total running count: 1604 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.492.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.494.714 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1705 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.507.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 201, total running count: 1605 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.507.666 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.509.298 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1706 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.522.820 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 202, total running count: 1606 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.523.195 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.524.881 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1707 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.538.385 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 203, total running count: 1607 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.538.731 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.540.350 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1708 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.553.933 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 204, total running count: 1608 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.554.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.555.961 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1709 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.569.317 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 205, total running count: 1609 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:52.569.680 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.571.476 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1710 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.584.697 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 206, total running count: 1610 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.585.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.586.944 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1711 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.600.244 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 207, total running count: 1611 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:52.600.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.602.568 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1712 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.615.708 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 208, total running count: 1612 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.616.058 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.618.205 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1713 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.631.191 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 209, total running count: 1613 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.631.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.634.084 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1714 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.646.637 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 210, total running count: 1614 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.646.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.648.602 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1715 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:52.662.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 211, total running count: 1615 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.662.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.665.370 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1716 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.677.641 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 212, total running count: 1616 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.678.032 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.679.757 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1717 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.693.138 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 213, total running count: 1617 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.693.501 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.695.089 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1718 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.708.545 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 214, total running count: 1618 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.708.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.710.761 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1719 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:52.724.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 215, total running count: 1619 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:52.724.558 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.726.280 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1720 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:52.739.599 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 216, total running count: 1620 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.739.987 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.741.835 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1721 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.755.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 217, total running count: 1621 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.755.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.757.309 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1722 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.770.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 218, total running count: 1622 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.770.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.772.885 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1723 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.786.020 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 219, total running count: 1623 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.786.378 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.788.607 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1724 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.801.484 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 220, total running count: 1624 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:52.801.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.804.309 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1725 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.816.775 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 221, total running count: 1625 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:52.817.147 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.818.977 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1726 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.832.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 222, total running count: 1626 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:52.832.563 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.834.499 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1727 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:52.847.750 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 223, total running count: 1627 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.848.121 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.850.314 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1728 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.863.134 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 224, total running count: 1628 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.863.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.865.181 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1729 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.879.779 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 225, total running count: 1629 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.880.145 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.881.795 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1730 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.896.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 226, total running count: 1630 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.896.516 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.898.698 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1731 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:52.911.856 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 227, total running count: 1631 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.912.205 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.914.467 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1732 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:52.927.319 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 228, total running count: 1632 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.927.694 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.930.264 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1733 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.942.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 229, total running count: 1633 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.943.269 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.944.837 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1734 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.958.300 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 230, total running count: 1634 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.958.637 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.960.678 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1735 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:52.973.922 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 231, total running count: 1635 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:52.974.278 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.976.482 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1736 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.989.432 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 232, total running count: 1636 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:52.989.792 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:52.992.154 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1737 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.004.837 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 233, total running count: 1637 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.005.183 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.007.510 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1738 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.020.352 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 234, total running count: 1638 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.020.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.023.230 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1739 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:53.035.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 235, total running count: 1639 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.036.301 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.038.018 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1740 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.051.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 236, total running count: 1640 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.051.691 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.053.821 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1741 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.066.705 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 237, total running count: 1641 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.067.096 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.069.387 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1742 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.081.997 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 238, total running count: 1642 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:53.082.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.084.970 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1743 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.097.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 239, total running count: 1643 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.097.843 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.100.421 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1744 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.112.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 240, total running count: 1644 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.113.271 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.115.900 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1745 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.128.498 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 241, total running count: 1645 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.128.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.130.587 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1746 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.143.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 242, total running count: 1646 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:53.144.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.146.340 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1747 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.159.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 243, total running count: 1647 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.160.111 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.161.943 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1748 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.175.375 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 244, total running count: 1648 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.175.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.177.856 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1749 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.190.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 245, total running count: 1649 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.191.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.193.515 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1750 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.206.343 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 246, total running count: 1650 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.206.722 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.209.266 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1751 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.221.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 247, total running count: 1651 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.222.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.223.760 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1752 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.237.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 248, total running count: 1652 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:53.237.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.239.695 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1753 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.252.755 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 249, total running count: 1653 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.253.116 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.255.381 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1754 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.268.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 250, total running count: 1654 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.268.698 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.270.903 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1755 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.283.795 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 251, total running count: 1655 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.284.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.286.597 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1756 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.299.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 252, total running count: 1656 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:53.299.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.302.260 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1757 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.314.738 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 253, total running count: 1657 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.315.121 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.317.778 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1758 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.330.118 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 254, total running count: 1658 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.330.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.332.153 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1759 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.345.423 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 255, total running count: 1659 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.345.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.347.999 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1760 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.360.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 256, total running count: 1660 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.361.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.362.722 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1761 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.375.874 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 257, total running count: 1661 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.376.379 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.378.293 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1762 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.391.442 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 258, total running count: 1662 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.391.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.393.844 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1763 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.407.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 259, total running count: 1663 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.407.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.409.764 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1764 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.422.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 260, total running count: 1664 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.423.176 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.425.601 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1765 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:53.439.271 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 261, total running count: 1665 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.439.670 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.442.189 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1766 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.455.391 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 262, total running count: 1666 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.455.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.457.800 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1767 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:53.471.131 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 263, total running count: 1667 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:53.471.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.473.259 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1768 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.486.681 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 264, total running count: 1668 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.487.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.488.740 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1769 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.502.338 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 265, total running count: 1669 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.502.713 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.504.410 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1770 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:53.517.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 266, total running count: 1670 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:53.518.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.520.119 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1771 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.533.458 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 267, total running count: 1671 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.533.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.536.080 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1772 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.549.001 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 268, total running count: 1672 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.549.345 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.551.614 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1773 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.564.453 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 269, total running count: 1673 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.564.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.567.247 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1774 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.579.969 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 270, total running count: 1674 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.580.350 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.582.704 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1775 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.595.678 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 271, total running count: 1675 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.596.053 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.598.201 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1776 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.611.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 272, total running count: 1676 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:53.611.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.613.828 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1777 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:53.626.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 273, total running count: 1677 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.626.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.628.518 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1778 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.641.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 274, total running count: 1678 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.642.371 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.644.321 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1779 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.657.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 275, total running count: 1679 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.657.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.659.966 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1780 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.673.053 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 276, total running count: 1680 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.673.413 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.675.389 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1781 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.688.558 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 277, total running count: 1681 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.688.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.691.108 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1782 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.705.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 278, total running count: 1682 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:53.705.525 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.707.776 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1783 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.722.032 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 279, total running count: 1683 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.722.392 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.724.442 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1784 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.737.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 280, total running count: 1684 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.738.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.739.878 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1785 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:53.753.558 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 281, total running count: 1685 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:53.753.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.756.431 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1786 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:53.769.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 282, total running count: 1686 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.769.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.771.923 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1787 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.784.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 283, total running count: 1687 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.785.267 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.787.397 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1788 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.800.613 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 284, total running count: 1688 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.800.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.803.112 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1789 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.816.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 285, total running count: 1689 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.816.661 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.818.941 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1790 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.831.808 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 286, total running count: 1690 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.832.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.833.728 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1791 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.847.420 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 287, total running count: 1691 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.847.759 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.849.353 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1792 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.862.882 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 288, total running count: 1692 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.863.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.864.804 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1793 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.878.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 289, total running count: 1693 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.878.781 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.880.581 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1794 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.893.756 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 290, total running count: 1694 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.894.107 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.896.388 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1795 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.909.415 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 291, total running count: 1695 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.909.791 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.912.155 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1796 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.924.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 292, total running count: 1696 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:53.925.165 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.926.707 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1797 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.940.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 293, total running count: 1697 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:53.940.704 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.942.543 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1798 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.955.755 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 294, total running count: 1698 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.956.105 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.958.430 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1799 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.971.412 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 295, total running count: 1699 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:53.971.766 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.974.210 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1800 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:53.987.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 296, total running count: 1700 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:53.987.750 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:53.989.781 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1801 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.002.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 297, total running count: 1701 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.003.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.005.334 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1802 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.018.619 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 298, total running count: 1702 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.018.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.020.860 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1803 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.034.168 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 299, total running count: 1703 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.034.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.036.500 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1804 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.049.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 300, total running count: 1704 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.050.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.052.027 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1805 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.064.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 301, total running count: 1705 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.065.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.067.683 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1806 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.080.489 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 302, total running count: 1706 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.080.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.083.408 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1807 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.095.998 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 303, total running count: 1707 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.096.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.097.929 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1808 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.111.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 304, total running count: 1708 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.111.885 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.113.607 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1809 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.126.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 305, total running count: 1709 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.127.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.129.465 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1810 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.142.460 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 306, total running count: 1710 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.142.833 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.145.368 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1811 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:54.158.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 307, total running count: 1711 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.158.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.160.969 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1812 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.173.608 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 308, total running count: 1712 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.174.000 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.175.546 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1813 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.189.109 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 309, total running count: 1713 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:54.189.466 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.191.305 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1814 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.204.510 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 310, total running count: 1714 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.204.899 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.206.905 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1815 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.220.165 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 311, total running count: 1715 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.220.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.222.491 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1816 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:54.235.589 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 312, total running count: 1716 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.236.002 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.238.206 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1817 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.251.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 313, total running count: 1717 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.251.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.254.042 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1818 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.266.719 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 314, total running count: 1718 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:54.267.087 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.269.725 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1819 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.282.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 315, total running count: 1719 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.282.722 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.285.304 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1820 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.297.788 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 316, total running count: 1720 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.298.132 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.299.717 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1821 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.313.303 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 317, total running count: 1721 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.313.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.315.364 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1822 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.328.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 318, total running count: 1722 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.329.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.331.217 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1823 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.344.458 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 319, total running count: 1723 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.344.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.346.938 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1824 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.359.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 320, total running count: 1724 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.360.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.362.417 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1825 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.375.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 321, total running count: 1725 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.375.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.378.166 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1826 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.391.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 322, total running count: 1726 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.391.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.394.164 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1827 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.407.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 323, total running count: 1727 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.407.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.409.639 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1828 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.422.587 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 324, total running count: 1728 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.422.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.425.215 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1829 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:54.438.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 325, total running count: 1729 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:54.438.596 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.440.793 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1830 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.454.045 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 326, total running count: 1730 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.454.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.456.547 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1831 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.469.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 327, total running count: 1731 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.470.247 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.472.446 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1832 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.485.533 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 328, total running count: 1732 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.486.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.488.570 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1833 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.501.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 329, total running count: 1733 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.501.898 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.504.158 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1834 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.517.335 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 330, total running count: 1734 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.517.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.520.097 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1835 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.533.158 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 331, total running count: 1735 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:54.533.522 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.536.164 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1836 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:54.548.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 332, total running count: 1736 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:54.549.244 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.550.920 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1837 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.564.611 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 333, total running count: 1737 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.564.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.566.742 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1838 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.580.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 334, total running count: 1738 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.580.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.582.714 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1839 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.595.992 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 335, total running count: 1739 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.596.378 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.598.794 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1840 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.611.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 336, total running count: 1740 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.611.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.614.573 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1841 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:54.627.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 337, total running count: 1741 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.627.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.630.255 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1842 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:54.643.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 338, total running count: 1742 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.643.611 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.645.994 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1843 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.658.926 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 339, total running count: 1743 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:54.659.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.661.905 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1844 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.674.707 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 340, total running count: 1744 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.675.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.676.686 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1845 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.690.441 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 341, total running count: 1745 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.690.811 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.692.625 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1846 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.706.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 342, total running count: 1746 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.706.844 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.708.634 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1847 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.722.358 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 343, total running count: 1747 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.722.730 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.724.456 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1848 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.738.134 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 344, total running count: 1748 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.738.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.740.219 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1849 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.754.015 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 345, total running count: 1749 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:54.754.399 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.757.045 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1850 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.769.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 346, total running count: 1750 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.770.177 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.772.792 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1851 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.785.681 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 347, total running count: 1751 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.786.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.787.777 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1852 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.801.407 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 348, total running count: 1752 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.801.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.803.794 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1853 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.817.260 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 349, total running count: 1753 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:54.817.622 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.819.497 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1854 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.833.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 350, total running count: 1754 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.834.408 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.836.395 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1855 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:54.850.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 351, total running count: 1755 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.851.236 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.853.504 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1856 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:54.866.777 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 352, total running count: 1756 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.868.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.870.727 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1857 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.883.922 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 353, total running count: 1757 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.884.444 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.886.494 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1858 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.899.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 354, total running count: 1758 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.900.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.902.462 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1859 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.915.840 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 355, total running count: 1759 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.916.287 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.918.261 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1860 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.931.683 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 356, total running count: 1760 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.932.079 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.933.755 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1861 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:54.947.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 357, total running count: 1761 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:54.948.208 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.950.818 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1862 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:54.963.439 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 358, total running count: 1762 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.964.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.966.721 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1863 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.979.750 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 359, total running count: 1763 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:54.980.129 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.982.529 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1864 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.996.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 360, total running count: 1764 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:54.996.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:54.998.942 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1865 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.013.314 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 361, total running count: 1765 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.013.679 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.015.559 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1866 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:55.029.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 362, total running count: 1766 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.030.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.032.113 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1867 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.045.596 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 363, total running count: 1767 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:55.045.982 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.048.623 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1868 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.061.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 364, total running count: 1768 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.062.184 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.064.529 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1869 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.077.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 365, total running count: 1769 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.078.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.080.496 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1870 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:55.093.680 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 366, total running count: 1770 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.094.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.096.104 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1871 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.109.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 367, total running count: 1771 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.110.072 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.112.625 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1872 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.125.521 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 368, total running count: 1772 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.125.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.128.173 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1873 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:55.141.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 369, total running count: 1773 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.141.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.143.728 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1874 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:55.157.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 370, total running count: 1774 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.157.535 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.159.430 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1875 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.173.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 371, total running count: 1775 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.173.496 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.176.072 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1876 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.191.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 372, total running count: 1776 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:55.191.871 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.194.155 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1877 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.207.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 373, total running count: 1777 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.207.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.209.977 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1878 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.222.720 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 374, total running count: 1778 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.223.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.225.642 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1879 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.238.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 375, total running count: 1779 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.238.739 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.241.336 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1880 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.253.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 376, total running count: 1780 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.254.348 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.257.024 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1881 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.269.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 377, total running count: 1781 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.269.813 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.271.624 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1882 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.284.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 378, total running count: 1782 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.285.243 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.287.745 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1883 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.300.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 379, total running count: 1783 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.301.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.303.459 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1884 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.316.302 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 380, total running count: 1784 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.316.668 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.318.980 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1885 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:55.331.888 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 381, total running count: 1785 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.332.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.334.482 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1886 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.348.225 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 382, total running count: 1786 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.348.608 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.350.605 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1887 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.364.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 383, total running count: 1787 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:55.365.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.367.737 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1888 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.380.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 384, total running count: 1788 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.380.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.383.567 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1889 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.396.163 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 385, total running count: 1789 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.396.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.398.116 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1890 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.411.898 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 386, total running count: 1790 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.412.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.415.169 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1891 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.428.136 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 387, total running count: 1791 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.429.183 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.431.171 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1892 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.444.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 388, total running count: 1792 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.444.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.446.839 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1893 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.460.277 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 389, total running count: 1793 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.460.985 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.463.444 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1894 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.476.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 390, total running count: 1794 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.476.833 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.479.132 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1895 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.492.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 391, total running count: 1795 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.492.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.494.928 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1896 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.507.837 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 392, total running count: 1796 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.508.205 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.510.934 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1897 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.525.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 393, total running count: 1797 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.525.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.527.631 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1898 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.542.425 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 394, total running count: 1798 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.542.806 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.544.399 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1899 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.558.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 395, total running count: 1799 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.558.785 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.560.370 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1900 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.574.396 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 396, total running count: 1800 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.574.749 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.576.447 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1901 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:55.590.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 397, total running count: 1801 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.590.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.592.387 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1902 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.606.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 398, total running count: 1802 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.606.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.609.116 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1903 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.622.314 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 399, total running count: 1803 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.622.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.624.853 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1904 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:55.638.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 400, total running count: 1804 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.638.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.640.866 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1905 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.654.191 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 401, total running count: 1805 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.654.570 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.656.851 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1906 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.670.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 402, total running count: 1806 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:55.671.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.673.670 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1907 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.686.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 403, total running count: 1807 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.687.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.689.834 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1908 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:55.703.037 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 404, total running count: 1808 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.703.795 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.705.532 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1909 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.719.114 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 405, total running count: 1809 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:55.719.848 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.722.371 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1910 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:55.735.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 406, total running count: 1810 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.735.899 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.737.999 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1911 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.751.330 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 407, total running count: 1811 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.751.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.753.777 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1912 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.767.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 408, total running count: 1812 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:55.768.150 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.770.823 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1913 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:55.783.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 409, total running count: 1813 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.783.985 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.786.328 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1914 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.799.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 410, total running count: 1814 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.799.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.801.965 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1915 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.815.153 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 411, total running count: 1815 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.815.728 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.817.994 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1916 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.831.000 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 412, total running count: 1816 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.831.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.833.835 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1917 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.846.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 413, total running count: 1817 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:55.847.247 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.849.434 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1918 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.862.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 414, total running count: 1818 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.863.044 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.865.200 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1919 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.878.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 415, total running count: 1819 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.878.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.880.905 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1920 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.894.192 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 416, total running count: 1820 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.895.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.897.781 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1921 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:55.910.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 417, total running count: 1821 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.910.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.913.396 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1922 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.926.169 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 418, total running count: 1822 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.926.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.929.006 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1923 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.941.795 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 419, total running count: 1823 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:55.942.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.944.911 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1924 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:55.957.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 420, total running count: 1824 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.958.243 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.960.822 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1925 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.973.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 421, total running count: 1825 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:55.974.054 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.976.409 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1926 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:55.989.412 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 422, total running count: 1826 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:55.989.949 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:55.992.163 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1927 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.005.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 423, total running count: 1827 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.005.636 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.008.106 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1928 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.020.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 424, total running count: 1828 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.021.382 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.023.903 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1929 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.036.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 425, total running count: 1829 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.037.045 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.038.629 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1930 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.052.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 426, total running count: 1830 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.053.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.055.736 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1931 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.068.601 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 427, total running count: 1831 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.069.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.071.304 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1932 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.084.522 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 428, total running count: 1832 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.085.413 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.087.971 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1933 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.100.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 429, total running count: 1833 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.101.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.103.819 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1934 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:56.116.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 430, total running count: 1834 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.116.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.118.526 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1935 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.132.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 431, total running count: 1835 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.132.473 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.134.377 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1936 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.147.674 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 432, total running count: 1836 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.148.051 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.150.117 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1937 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:56.163.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 433, total running count: 1837 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.163.660 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.166.129 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1938 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.179.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 434, total running count: 1838 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.179.407 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.181.812 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1939 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.194.688 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 435, total running count: 1839 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:56.195.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.197.475 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1940 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:56.210.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 436, total running count: 1840 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.210.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.213.173 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1941 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.226.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 437, total running count: 1841 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.226.915 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.228.830 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1942 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:56.242.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 438, total running count: 1842 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.242.711 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.244.485 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1943 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.257.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 439, total running count: 1843 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.258.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.260.338 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1944 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.273.498 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 440, total running count: 1844 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.273.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.276.128 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1945 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.289.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 441, total running count: 1845 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.289.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.292.222 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1946 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.304.727 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 442, total running count: 1846 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.305.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.307.045 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1947 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.320.620 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 443, total running count: 1847 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.320.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.323.830 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1948 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:56.336.235 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 444, total running count: 1848 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.336.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.338.939 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1949 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.351.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 445, total running count: 1849 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.352.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.355.060 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1950 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.368.044 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 446, total running count: 1850 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.368.508 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.370.808 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1951 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.383.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 447, total running count: 1851 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.384.204 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.386.383 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1952 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.399.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 448, total running count: 1852 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.399.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.401.888 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1953 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:56.415.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 449, total running count: 1853 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.415.817 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.417.645 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1954 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.431.239 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 450, total running count: 1854 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:56.431.629 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.433.830 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1955 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.446.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 451, total running count: 1855 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.447.300 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.449.551 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1956 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:56.462.453 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 452, total running count: 1856 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.462.839 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.465.391 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1957 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:56.477.900 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 453, total running count: 1857 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.478.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.480.940 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1958 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.494.016 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 454, total running count: 1858 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.494.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.496.948 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1959 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.509.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 455, total running count: 1859 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.510.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.512.429 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1960 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.525.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 456, total running count: 1860 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.525.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.527.996 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1961 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:56.540.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 457, total running count: 1861 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:56.541.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.543.981 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1962 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.556.645 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 458, total running count: 1862 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.557.033 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.558.852 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1963 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.572.260 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 459, total running count: 1863 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.572.614 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.574.294 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1964 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.587.689 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 460, total running count: 1864 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.588.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.590.108 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1965 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.603.400 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 461, total running count: 1865 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.603.770 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.605.775 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1966 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.618.946 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 462, total running count: 1866 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.619.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.621.578 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1967 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:56.634.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 463, total running count: 1867 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.635.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.637.284 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1968 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.650.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 464, total running count: 1868 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.650.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.653.071 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1969 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.666.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 465, total running count: 1869 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.666.407 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.669.245 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1970 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.681.601 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 466, total running count: 1870 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.681.968 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.683.540 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1971 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.697.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 467, total running count: 1871 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.697.533 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.699.182 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1972 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.712.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 468, total running count: 1872 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:40:56.713.220 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_1 execution count: 4, execution time: 7324.91 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:40:56.713.379 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_1 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:40:56.713.481 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty epoch: 4 step: 468, loss is 0.5957637429237366 [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:40:56.714.506 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.h:76] ContinueSend] continue send at the beginning of the epoch [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:40:56.715.657 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:40:56.715.720 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:40:56.715.759 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_1 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.716.121 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.718.305 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1973 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.731.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 1, total running count: 1873 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.731.948 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.733.794 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1974 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.747.375 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 2, total running count: 1874 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.747.731 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.749.529 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1975 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.763.054 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 3, total running count: 1875 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.763.434 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.765.647 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1976 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.778.615 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 4, total running count: 1876 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:56.778.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.781.652 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1977 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.794.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 5, total running count: 1877 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.794.403 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.796.099 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1978 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.809.476 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 6, total running count: 1878 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.809.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.811.706 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1979 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:56.825.001 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 7, total running count: 1879 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.825.458 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.827.498 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1980 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.840.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 8, total running count: 1880 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.840.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.842.930 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1981 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.856.378 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 9, total running count: 1881 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.857.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.860.193 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1982 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.872.915 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 10, total running count: 1882 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.873.627 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.875.634 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1983 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.888.388 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 11, total running count: 1883 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.888.833 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.891.234 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1984 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.903.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 12, total running count: 1884 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.903.899 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.905.617 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1985 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.918.700 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 13, total running count: 1885 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.919.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.921.246 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1986 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.933.816 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 14, total running count: 1886 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.934.567 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.937.176 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1987 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:56.949.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 15, total running count: 1887 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.949.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.951.866 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1988 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.964.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 16, total running count: 1888 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.965.392 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.967.323 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1989 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.980.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 17, total running count: 1889 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:56.980.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.983.117 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1990 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:56.995.514 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 18, total running count: 1890 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:56.996.582 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:56.998.914 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1991 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.011.106 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 19, total running count: 1891 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.012.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.014.848 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1992 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.027.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 20, total running count: 1892 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.027.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.029.407 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1993 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.042.109 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 21, total running count: 1893 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.042.889 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.045.136 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1994 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.057.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 22, total running count: 1894 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.058.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.060.758 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1995 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.073.177 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 23, total running count: 1895 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.074.264 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.076.560 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1996 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.089.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 24, total running count: 1896 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.089.973 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.091.951 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1997 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.104.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 25, total running count: 1897 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.105.525 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.107.400 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1998 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.120.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 26, total running count: 1898 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.121.330 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.122.884 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1999 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.136.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 27, total running count: 1899 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.136.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.139.496 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2000 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.151.570 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 28, total running count: 1900 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:57.152.691 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.154.933 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2001 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.167.400 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 29, total running count: 1901 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.168.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.170.431 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2002 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.183.128 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 30, total running count: 1902 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.183.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.185.294 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2003 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.198.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 31, total running count: 1903 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.199.099 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.201.128 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2004 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.213.850 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 32, total running count: 1904 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.214.586 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.216.662 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2005 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.229.214 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 33, total running count: 1905 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.229.898 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.232.134 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2006 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.244.630 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 34, total running count: 1906 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:57.245.871 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.248.018 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2007 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.260.758 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 35, total running count: 1907 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.261.612 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.263.928 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2008 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:57.276.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 36, total running count: 1908 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.277.219 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.279.455 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2009 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.291.779 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 37, total running count: 1909 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.292.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.295.014 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2010 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.307.808 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 38, total running count: 1910 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.308.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.310.480 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2011 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.323.311 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 39, total running count: 1911 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.324.349 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.326.075 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2012 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.338.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 40, total running count: 1912 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:57.339.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.341.860 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2013 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.354.721 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 41, total running count: 1913 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.355.909 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.357.601 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2014 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.370.754 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 42, total running count: 1914 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:57.371.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.374.534 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2015 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.386.598 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 43, total running count: 1915 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.387.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.390.331 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2016 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.402.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 44, total running count: 1916 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.403.210 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.405.029 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2017 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.417.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 45, total running count: 1917 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.418.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.420.662 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2018 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.433.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 46, total running count: 1918 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.434.059 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.436.442 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2019 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.448.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 47, total running count: 1919 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.449.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.451.192 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2020 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.464.231 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 48, total running count: 1920 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.465.121 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.466.762 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2021 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.479.759 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 49, total running count: 1921 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.480.793 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.483.388 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2022 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.495.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 50, total running count: 1922 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.496.713 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.498.863 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2023 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.511.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 51, total running count: 1923 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.512.278 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.514.579 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2024 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.527.059 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 52, total running count: 1924 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.527.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.530.591 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2025 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.542.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 53, total running count: 1925 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.543.374 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.545.042 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2026 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.557.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 54, total running count: 1926 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.558.937 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.560.556 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2027 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.573.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 55, total running count: 1927 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.574.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.576.421 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2028 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.588.831 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 56, total running count: 1928 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.590.203 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.592.266 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2029 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.605.028 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 57, total running count: 1929 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.605.820 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.607.714 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2030 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.620.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 58, total running count: 1930 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.621.388 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.623.361 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2031 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.636.033 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 59, total running count: 1931 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.637.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.639.189 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2032 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.651.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 60, total running count: 1932 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.652.740 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.655.049 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2033 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.667.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 61, total running count: 1933 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.668.158 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.670.702 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2034 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.682.861 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 62, total running count: 1934 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.683.776 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.685.392 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2035 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.698.402 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 63, total running count: 1935 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.699.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.701.212 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2036 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.713.944 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 64, total running count: 1936 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.714.687 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.716.649 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2037 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.729.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 65, total running count: 1937 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.730.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.732.397 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2038 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.744.859 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 66, total running count: 1938 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.746.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.748.101 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2039 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.760.973 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 67, total running count: 1939 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.762.192 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.764.055 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2040 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.776.550 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 68, total running count: 1940 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.777.615 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.779.527 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2041 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.791.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 69, total running count: 1941 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.793.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.795.216 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2042 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.807.689 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 70, total running count: 1942 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.808.794 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.810.806 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2043 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.823.458 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 71, total running count: 1943 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.824.421 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.826.801 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2044 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.838.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 72, total running count: 1944 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.840.024 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.842.513 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2045 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.854.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 73, total running count: 1945 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.855.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.857.368 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2046 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.869.948 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 74, total running count: 1946 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.870.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.873.248 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2047 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.885.508 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 75, total running count: 1947 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.888.231 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.890.126 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2048 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.902.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 76, total running count: 1948 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.903.187 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.905.902 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2049 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.917.799 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 77, total running count: 1949 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.918.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.920.615 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2050 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.932.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 78, total running count: 1950 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.934.219 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.936.416 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2051 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:57.948.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 79, total running count: 1951 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.949.935 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.952.388 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2052 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.964.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 80, total running count: 1952 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.965.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.968.025 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2053 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:57.980.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 81, total running count: 1953 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:57.980.785 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.982.395 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2054 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.995.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 82, total running count: 1954 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:57.996.403 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:57.998.295 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2055 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.010.926 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 83, total running count: 1955 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.012.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.014.186 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2056 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.026.674 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 84, total running count: 1956 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.027.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.029.769 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2057 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.042.184 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 85, total running count: 1957 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.042.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.045.452 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2058 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.057.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 86, total running count: 1958 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.058.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.060.149 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2059 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.073.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 87, total running count: 1959 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.074.120 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.075.799 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2060 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.088.766 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 88, total running count: 1960 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.089.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.092.320 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2061 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.104.222 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 89, total running count: 1961 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.105.107 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.106.786 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2062 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.119.755 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 90, total running count: 1962 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.120.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.123.304 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2063 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.135.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 91, total running count: 1963 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.136.120 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.137.655 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2064 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.150.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 92, total running count: 1964 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.151.462 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.153.350 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2065 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.166.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 93, total running count: 1965 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.166.926 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.169.253 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2066 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.181.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 94, total running count: 1966 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.182.493 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.184.940 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2067 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.197.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 95, total running count: 1967 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.198.154 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.200.426 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2068 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.212.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 96, total running count: 1968 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.213.440 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.215.928 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2069 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.227.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 97, total running count: 1969 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.228.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.231.407 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2070 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.243.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 98, total running count: 1970 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.244.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.246.840 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2071 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.259.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 99, total running count: 1971 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.260.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.262.545 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2072 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.274.719 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 100, total running count: 1972 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.275.479 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.277.181 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2073 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.289.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 101, total running count: 1973 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.290.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.292.981 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2074 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.305.883 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 102, total running count: 1974 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.306.922 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.308.863 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2075 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.321.556 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 103, total running count: 1975 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.322.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.324.928 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2076 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.336.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 104, total running count: 1976 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.337.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.340.481 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2077 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.352.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 105, total running count: 1977 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.353.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.355.900 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2078 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.368.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 106, total running count: 1978 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.368.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.371.679 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2079 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.383.584 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 107, total running count: 1979 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.384.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.386.297 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2080 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.399.019 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 108, total running count: 1980 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.399.942 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.401.831 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2081 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.414.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 109, total running count: 1981 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.415.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.417.582 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2082 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.430.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 110, total running count: 1982 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.431.128 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.433.350 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2083 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.445.594 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 111, total running count: 1983 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.446.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.449.172 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2084 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.461.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 112, total running count: 1984 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.462.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.464.636 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2085 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.476.620 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 113, total running count: 1985 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.477.530 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.479.161 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2086 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.492.199 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 114, total running count: 1986 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.493.248 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.494.858 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2087 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.507.815 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 115, total running count: 1987 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.508.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.510.770 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2088 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.523.097 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 116, total running count: 1988 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.523.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.526.563 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2089 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.538.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 117, total running count: 1989 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.539.318 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.540.969 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2090 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.554.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 118, total running count: 1990 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.554.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.556.925 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2091 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.569.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 119, total running count: 1991 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.570.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.572.655 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2092 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.584.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 120, total running count: 1992 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.585.584 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.587.335 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2093 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.600.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 121, total running count: 1993 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.601.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.603.257 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2094 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.615.945 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 122, total running count: 1994 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.616.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.618.948 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2095 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.631.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 123, total running count: 1995 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.632.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.634.553 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2096 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.647.018 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 124, total running count: 1996 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.648.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.650.453 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2097 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.662.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 125, total running count: 1997 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.663.543 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.666.205 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2098 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.677.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 126, total running count: 1998 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.679.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.681.783 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2099 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.693.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 127, total running count: 1999 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.694.879 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.696.391 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2100 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.709.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 128, total running count: 2000 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.710.419 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.712.338 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2101 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:58.724.908 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 129, total running count: 2001 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.725.968 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.727.959 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2102 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.740.642 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 130, total running count: 2002 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.741.587 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.743.597 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2103 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.756.150 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 131, total running count: 2003 [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:40:58.756.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.759.246 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2104 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.771.651 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 132, total running count: 2004 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.772.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.773.833 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2105 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.786.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 133, total running count: 2005 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.787.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.789.486 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2106 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.802.495 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 134, total running count: 2006 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.803.354 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.805.997 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2107 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.818.169 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 135, total running count: 2007 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.818.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.820.461 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2108 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.833.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 136, total running count: 2008 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.834.000 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.835.871 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2109 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.848.828 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 137, total running count: 2009 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.849.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.851.247 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2110 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.864.018 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 138, total running count: 2010 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.864.831 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.866.736 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2111 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.879.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 139, total running count: 2011 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.880.135 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.882.269 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2112 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.894.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 140, total running count: 2012 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.895.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.897.785 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2113 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.910.108 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 141, total running count: 2013 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.911.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.913.633 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2114 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.926.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 142, total running count: 2014 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.926.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.929.544 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2115 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.941.485 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 143, total running count: 2015 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.942.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.945.099 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2116 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.956.813 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 144, total running count: 2016 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.958.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.960.602 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2117 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:58.972.843 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 145, total running count: 2017 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:58.973.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.974.961 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2118 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:58.987.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 146, total running count: 2018 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:58.988.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:58.990.697 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2119 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.003.588 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 147, total running count: 2019 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.004.325 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.006.469 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2120 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.019.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 148, total running count: 2020 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.019.744 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.022.257 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2121 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.034.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 149, total running count: 2021 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.035.219 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.037.635 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2122 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.049.700 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 150, total running count: 2022 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.050.827 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.053.181 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2123 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.065.623 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 151, total running count: 2023 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.066.407 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.068.866 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2124 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.081.071 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 152, total running count: 2024 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.082.128 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.083.686 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2125 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.096.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 153, total running count: 2025 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.097.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.099.358 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2126 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.112.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 154, total running count: 2026 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.112.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.114.710 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2127 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.127.647 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 155, total running count: 2027 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.128.348 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.130.338 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2128 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.143.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 156, total running count: 2028 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.143.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.145.789 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2129 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.158.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 157, total running count: 2029 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.159.286 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.161.395 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2130 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.174.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 158, total running count: 2030 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.174.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.176.973 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2131 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.189.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 159, total running count: 2031 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.190.318 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.192.917 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2132 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.205.009 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 160, total running count: 2032 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.205.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.208.378 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2133 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.220.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 161, total running count: 2033 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.221.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.223.880 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2134 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.235.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 162, total running count: 2034 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.236.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.239.253 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2135 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.251.434 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 163, total running count: 2035 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.252.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.254.805 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2136 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.266.947 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 164, total running count: 2036 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.267.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.270.310 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2137 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.282.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 165, total running count: 2037 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.283.241 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.285.680 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2138 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.297.988 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 166, total running count: 2038 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.298.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.301.179 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2139 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.313.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 167, total running count: 2039 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.314.474 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.316.574 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2140 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.329.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 168, total running count: 2040 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.329.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.332.064 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2141 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.344.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 169, total running count: 2041 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.345.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.347.626 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2142 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.360.222 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 170, total running count: 2042 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.360.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.362.958 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2143 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.375.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 171, total running count: 2043 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.376.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.378.452 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2144 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.391.178 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 172, total running count: 2044 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.391.845 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.394.547 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2145 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.406.594 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 173, total running count: 2045 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.407.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.409.056 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2146 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.421.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 174, total running count: 2046 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.423.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.424.633 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2147 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.437.678 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 175, total running count: 2047 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.438.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.440.138 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2148 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.453.139 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 176, total running count: 2048 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.453.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.455.680 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2149 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.468.679 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 177, total running count: 2049 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.469.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.471.222 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2150 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.483.905 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 178, total running count: 2050 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.484.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.486.810 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2151 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.499.381 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 179, total running count: 2051 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.500.232 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.502.342 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2152 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.514.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 180, total running count: 2052 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.515.843 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.517.966 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2153 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.530.372 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 181, total running count: 2053 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.531.495 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.534.123 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2154 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.545.992 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 182, total running count: 2054 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.547.292 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.549.797 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2155 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.561.725 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 183, total running count: 2055 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.563.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.565.227 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2156 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.577.724 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 184, total running count: 2056 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.578.928 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.580.897 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2157 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.593.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 185, total running count: 2057 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.594.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.596.615 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2158 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.609.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 186, total running count: 2058 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.610.241 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.612.466 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2159 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.624.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 187, total running count: 2059 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.625.756 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.628.097 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2160 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.640.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 188, total running count: 2060 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.641.371 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.643.729 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2161 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.656.169 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 189, total running count: 2061 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.657.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.659.367 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2162 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.671.544 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 190, total running count: 2062 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.672.815 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.675.250 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2163 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.687.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 191, total running count: 2063 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.688.415 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.690.764 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2164 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.702.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 192, total running count: 2064 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.703.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.706.222 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2165 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.718.733 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 193, total running count: 2065 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.719.721 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.721.765 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2166 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.734.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 194, total running count: 2066 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.735.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.737.453 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2167 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.749.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 195, total running count: 2067 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.750.737 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.753.270 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2168 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.765.478 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 196, total running count: 2068 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.766.235 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.768.893 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2169 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.781.086 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 197, total running count: 2069 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.781.601 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.783.276 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2170 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.796.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 198, total running count: 2070 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.797.226 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.799.038 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2171 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.811.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 199, total running count: 2071 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.812.707 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.814.995 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2172 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.827.259 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 200, total running count: 2072 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.828.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.830.802 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2173 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.842.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 201, total running count: 2073 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.843.666 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.846.568 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2174 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.858.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 202, total running count: 2074 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.859.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.861.243 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2175 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.873.844 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 203, total running count: 2075 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.874.746 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.876.904 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2176 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.889.361 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 204, total running count: 2076 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.890.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.892.462 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2177 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.905.046 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 205, total running count: 2077 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.905.717 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.907.902 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2178 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.920.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 206, total running count: 2078 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.921.204 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.923.624 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2179 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.935.679 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 207, total running count: 2079 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:40:59.936.680 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.938.289 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2180 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.951.324 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 208, total running count: 2080 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:40:59.952.135 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.954.380 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2181 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.966.756 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 209, total running count: 2081 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.967.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.969.966 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2182 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:40:59.982.192 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 210, total running count: 2082 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.983.352 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:40:59.985.917 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2183 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.997.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 211, total running count: 2083 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:40:59.999.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.000.621 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2184 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.013.640 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 212, total running count: 2084 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.014.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.016.177 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2185 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.029.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 213, total running count: 2085 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.030.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.031.880 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2186 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.044.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 214, total running count: 2086 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.045.833 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.047.665 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2187 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.060.486 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 215, total running count: 2087 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.061.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.063.465 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2188 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.076.569 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 216, total running count: 2088 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.077.419 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.080.100 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2189 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.092.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 217, total running count: 2089 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.093.121 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.095.750 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2190 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.107.864 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 218, total running count: 2090 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.108.717 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.110.665 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2191 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.123.439 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 219, total running count: 2091 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.124.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.126.348 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2192 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.139.007 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 220, total running count: 2092 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.139.878 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.141.963 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2193 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.154.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 221, total running count: 2093 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.155.938 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.157.759 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2194 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.170.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 222, total running count: 2094 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.171.545 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.173.551 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2195 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.186.058 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 223, total running count: 2095 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.187.182 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.189.152 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2196 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.201.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 224, total running count: 2096 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.202.806 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.204.590 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2197 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.217.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 225, total running count: 2097 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.218.455 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.220.102 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2198 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.232.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 226, total running count: 2098 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.233.871 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.235.785 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2199 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.248.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 227, total running count: 2099 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.249.481 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.251.767 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2200 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.264.392 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 228, total running count: 2100 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.265.292 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.267.410 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2201 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.280.013 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 229, total running count: 2101 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.281.092 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.282.890 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2202 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.295.539 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 230, total running count: 2102 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.296.730 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.298.482 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2203 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.311.204 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 231, total running count: 2103 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.312.416 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.314.259 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2204 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.326.997 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 232, total running count: 2104 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.328.052 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.330.256 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2205 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.342.916 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 233, total running count: 2105 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.343.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.345.876 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2206 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.358.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 234, total running count: 2106 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.359.416 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.361.677 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2207 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.374.044 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 235, total running count: 2107 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.375.115 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.377.453 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2208 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.389.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 236, total running count: 2108 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.390.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.393.320 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2209 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.405.539 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 237, total running count: 2109 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.406.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.409.001 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2210 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.422.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 238, total running count: 2110 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.422.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.424.868 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2211 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.437.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 239, total running count: 2111 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.437.758 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.439.456 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2212 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.452.362 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 240, total running count: 2112 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.453.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.455.090 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2213 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.467.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 241, total running count: 2113 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.468.740 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.470.542 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2214 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.483.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 242, total running count: 2114 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.484.335 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.486.540 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2215 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.498.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 243, total running count: 2115 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.499.861 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.502.400 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2216 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.514.374 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 244, total running count: 2116 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.515.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.517.259 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2217 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.530.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 245, total running count: 2117 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.531.016 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.532.902 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2218 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.545.670 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 246, total running count: 2118 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.546.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.548.935 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2219 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.561.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 247, total running count: 2119 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.562.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.563.731 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2220 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.576.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 248, total running count: 2120 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.577.547 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.579.318 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2221 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.592.150 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 249, total running count: 2121 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.593.091 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.594.917 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2222 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.607.791 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 250, total running count: 2122 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.608.533 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.610.777 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2223 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.623.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 251, total running count: 2123 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.623.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.625.472 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2224 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.638.485 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 252, total running count: 2124 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.639.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.641.132 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2225 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.653.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 253, total running count: 2125 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.654.962 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.656.588 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2226 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.669.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 254, total running count: 2126 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.670.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.672.050 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2227 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.684.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 255, total running count: 2127 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.685.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.687.474 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2228 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.700.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 256, total running count: 2128 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.701.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.703.993 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2229 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.716.183 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 257, total running count: 2129 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.717.264 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.719.376 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2230 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.731.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 258, total running count: 2130 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.733.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.734.931 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2231 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.747.930 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 259, total running count: 2131 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.748.796 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.750.807 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2232 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.763.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 260, total running count: 2132 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.764.262 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.766.814 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2233 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.779.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 261, total running count: 2133 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.780.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.782.354 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2234 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.794.698 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 262, total running count: 2134 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.795.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.798.043 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2235 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.810.176 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 263, total running count: 2135 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.811.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.813.742 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2236 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.826.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 264, total running count: 2136 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.826.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.828.406 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2237 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.841.572 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 265, total running count: 2137 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.842.576 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.845.005 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2238 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.856.930 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 266, total running count: 2138 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.858.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.860.429 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2239 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.872.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 267, total running count: 2139 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.873.601 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.875.877 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2240 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.888.362 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 268, total running count: 2140 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.889.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.891.303 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2241 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.903.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 269, total running count: 2141 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.905.039 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.906.736 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2242 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.919.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 270, total running count: 2142 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.920.637 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.922.325 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2243 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.935.356 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 271, total running count: 2143 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:00.936.176 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.938.079 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2244 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.950.737 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 272, total running count: 2144 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.951.611 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.953.780 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2245 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.966.169 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 273, total running count: 2145 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:00.967.199 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.969.262 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2246 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:00.981.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 274, total running count: 2146 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.982.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:00.984.987 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2247 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.997.314 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 275, total running count: 2147 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:00.998.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.000.969 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2248 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.013.343 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 276, total running count: 2148 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.014.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.016.480 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2249 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.028.865 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 277, total running count: 2149 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.029.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.031.857 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2250 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.044.392 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 278, total running count: 2150 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.045.131 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.047.398 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2251 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.059.829 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 279, total running count: 2151 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.060.708 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.062.945 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2252 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.075.235 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 280, total running count: 2152 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.076.311 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.078.498 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2253 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.090.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 281, total running count: 2153 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.091.828 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.094.479 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2254 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.106.387 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 282, total running count: 2154 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.107.251 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.109.157 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2255 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.121.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 283, total running count: 2155 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.122.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.124.951 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2256 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.137.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 284, total running count: 2156 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.138.372 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.140.602 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2257 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.153.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 285, total running count: 2157 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.154.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.156.425 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2258 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.168.700 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 286, total running count: 2158 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.169.657 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.171.989 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2259 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.184.218 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 287, total running count: 2159 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.185.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.187.559 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2260 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.199.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 288, total running count: 2160 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.200.719 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.203.370 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2261 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.215.471 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 289, total running count: 2161 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.216.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.217.908 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2262 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.230.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 290, total running count: 2162 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.231.765 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.233.539 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2263 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.246.350 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 291, total running count: 2163 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.247.304 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.249.021 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2264 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.261.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 292, total running count: 2164 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.262.719 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.264.675 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2265 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.277.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 293, total running count: 2165 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.278.302 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.280.518 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2266 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.293.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 294, total running count: 2166 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.293.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.296.307 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2267 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.308.361 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 295, total running count: 2167 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.309.354 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.311.996 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2268 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.324.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 296, total running count: 2168 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.324.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.326.578 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2269 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.339.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 297, total running count: 2169 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.340.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.342.552 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2270 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.355.168 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 298, total running count: 2170 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.355.865 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.358.247 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2271 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.370.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 299, total running count: 2171 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.371.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.373.792 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2272 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.385.860 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 300, total running count: 2172 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.386.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.389.311 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2273 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.401.580 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 301, total running count: 2173 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.402.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.404.060 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2274 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.417.122 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 302, total running count: 2174 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.418.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.419.876 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2275 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.432.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 303, total running count: 2175 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.433.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.435.567 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2276 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.448.640 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 304, total running count: 2176 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.449.281 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.451.459 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2277 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.463.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 305, total running count: 2177 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.464.768 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.467.180 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2278 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.479.423 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 306, total running count: 2178 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.480.096 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.482.790 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2279 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.494.568 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 307, total running count: 2179 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.495.493 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.497.258 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2280 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.510.108 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 308, total running count: 2180 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.510.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.513.235 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2281 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.525.615 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 309, total running count: 2181 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.526.427 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.528.950 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2282 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.540.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 310, total running count: 2182 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.541.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.543.441 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2283 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.556.392 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 311, total running count: 2183 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.557.290 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.559.016 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2284 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.571.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 312, total running count: 2184 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.572.602 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.574.717 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2285 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.587.151 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 313, total running count: 2185 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.588.120 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.590.568 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2286 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.602.782 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 314, total running count: 2186 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.603.550 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.606.277 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2287 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.618.473 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 315, total running count: 2187 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.618.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.620.883 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2288 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.633.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 316, total running count: 2188 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.634.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.636.869 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2289 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.649.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 317, total running count: 2189 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.649.812 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.651.378 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2290 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.664.493 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 318, total running count: 2190 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.665.187 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.667.220 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2291 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.679.735 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 319, total running count: 2191 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.680.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.683.118 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2292 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.695.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 320, total running count: 2192 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.695.937 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.697.715 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2293 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.710.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 321, total running count: 2193 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.711.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.713.440 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2294 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.726.389 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 322, total running count: 2194 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.727.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.729.461 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2295 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.741.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 323, total running count: 2195 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.742.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.744.993 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2296 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.757.035 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 324, total running count: 2196 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.757.796 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.759.537 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2297 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.772.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 325, total running count: 2197 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.773.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.775.238 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2298 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.788.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 326, total running count: 2198 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.788.908 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.791.198 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2299 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.803.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 327, total running count: 2199 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.804.498 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.806.828 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2300 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.818.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 328, total running count: 2200 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.819.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.822.445 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2301 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.834.443 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 329, total running count: 2201 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.835.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.837.268 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2302 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.850.149 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 330, total running count: 2202 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.851.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.852.939 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2303 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.865.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 331, total running count: 2203 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.866.686 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.868.567 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2304 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.881.265 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 332, total running count: 2204 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.882.183 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.884.108 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2305 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.896.937 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 333, total running count: 2205 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.897.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.899.592 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2306 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.912.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 334, total running count: 2206 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.913.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.915.419 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2307 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.928.300 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 335, total running count: 2207 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.928.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.931.178 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2308 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.943.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 336, total running count: 2208 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:01.944.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.946.833 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2309 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.959.350 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 337, total running count: 2209 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.960.225 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.962.328 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2310 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.974.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 338, total running count: 2210 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:01.975.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.977.835 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2311 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:01.990.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 339, total running count: 2211 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:01.991.653 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:01.993.652 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2312 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.006.100 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 340, total running count: 2212 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.007.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.009.495 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2313 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.021.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 341, total running count: 2213 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.022.890 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.025.147 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2314 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.037.503 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 342, total running count: 2214 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.038.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.040.874 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2315 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.052.820 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 343, total running count: 2215 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.053.724 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.055.598 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2316 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.068.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 344, total running count: 2216 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.069.232 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.071.307 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2317 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.083.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 345, total running count: 2217 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.085.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.086.908 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2318 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.099.660 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 346, total running count: 2218 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.100.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.102.638 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2319 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.115.444 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 347, total running count: 2219 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.116.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.118.484 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2320 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.130.960 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 348, total running count: 2220 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.131.766 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.134.179 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2321 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.146.479 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 349, total running count: 2221 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.147.286 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.149.895 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2322 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.161.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 350, total running count: 2222 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.162.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.165.628 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2323 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.177.444 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 351, total running count: 2223 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.178.606 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.180.409 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2324 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.193.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 352, total running count: 2224 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.194.289 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.195.872 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2325 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.209.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 353, total running count: 2225 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.209.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.211.642 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2326 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.224.419 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 354, total running count: 2226 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.225.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.227.696 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2327 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.240.162 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 355, total running count: 2227 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.240.908 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.243.407 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2328 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.255.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 356, total running count: 2228 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.256.361 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.258.865 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2329 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.271.084 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 357, total running count: 2229 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.271.903 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.274.545 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2330 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.286.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 358, total running count: 2230 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.287.749 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.290.432 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2331 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.302.392 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 359, total running count: 2231 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.303.145 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.305.152 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2332 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.317.554 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 360, total running count: 2232 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.318.731 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.320.932 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2333 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.333.468 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 361, total running count: 2233 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.334.382 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.336.900 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2334 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.348.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 362, total running count: 2234 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.349.718 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.351.417 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2335 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.364.381 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 363, total running count: 2235 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.364.978 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.367.006 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2336 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.379.532 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 364, total running count: 2236 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.380.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.382.791 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2337 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.395.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 365, total running count: 2237 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.396.111 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.398.559 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2338 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.410.721 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 366, total running count: 2238 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.411.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.414.019 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2339 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.426.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 367, total running count: 2239 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.427.130 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.429.720 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2340 batch(es) to device, channel name: 954c4ab6-af69-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1affd0f0,python):2024-01-10-11:41:02.429.790 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:596] SendDataToAscend] ExecutionTree finished. Device queue sent number of batches: 2340 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.441.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 368, total running count: 2240 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.442.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.457.124 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 369, total running count: 2241 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.457.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.472.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 370, total running count: 2242 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.473.606 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.488.213 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 371, total running count: 2243 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.489.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.504.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 372, total running count: 2244 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.505.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.519.707 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 373, total running count: 2245 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.520.596 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.535.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 374, total running count: 2246 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.536.127 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.550.632 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 375, total running count: 2247 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.551.683 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.566.303 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 376, total running count: 2248 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.567.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.583.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 377, total running count: 2249 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.584.078 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.598.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 378, total running count: 2250 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.599.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.613.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 379, total running count: 2251 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.614.782 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.629.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 380, total running count: 2252 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.630.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.644.799 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 381, total running count: 2253 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.646.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.660.861 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 382, total running count: 2254 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.662.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.676.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 383, total running count: 2255 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.677.636 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.692.165 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 384, total running count: 2256 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.693.131 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.707.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 385, total running count: 2257 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.708.775 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.723.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 386, total running count: 2258 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.724.349 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.739.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 387, total running count: 2259 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.739.816 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.754.731 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 388, total running count: 2260 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.755.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.769.945 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 389, total running count: 2261 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.771.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.785.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 390, total running count: 2262 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.786.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.800.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 391, total running count: 2263 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.801.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.816.495 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 392, total running count: 2264 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.817.480 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.831.971 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 393, total running count: 2265 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.833.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.847.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 394, total running count: 2266 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.848.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.863.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 395, total running count: 2267 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.864.289 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.878.698 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 396, total running count: 2268 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.879.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.894.498 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 397, total running count: 2269 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.895.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.909.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 398, total running count: 2270 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.910.704 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.925.265 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 399, total running count: 2271 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.927.099 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.941.677 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 400, total running count: 2272 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.944.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.959.191 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 401, total running count: 2273 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:02.960.416 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.974.925 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 402, total running count: 2274 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:02.976.009 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:02.990.547 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 403, total running count: 2275 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:02.991.700 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.006.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 404, total running count: 2276 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.007.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.022.412 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 405, total running count: 2277 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.023.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.038.024 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 406, total running count: 2278 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.038.887 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.053.724 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 407, total running count: 2279 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.054.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.069.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 408, total running count: 2280 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.070.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.084.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 409, total running count: 2281 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.085.804 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.100.308 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 410, total running count: 2282 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.101.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.116.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 411, total running count: 2283 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.117.216 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.131.695 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 412, total running count: 2284 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.132.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.147.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 413, total running count: 2285 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.148.225 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.163.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 414, total running count: 2286 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.163.944 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.178.539 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 415, total running count: 2287 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.179.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.193.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 416, total running count: 2288 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.194.680 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.209.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 417, total running count: 2289 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.210.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.224.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 418, total running count: 2290 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.225.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:41:03.240.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 419, total running count: 2291 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.241.043 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.255.658 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 420, total running count: 2292 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.256.408 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.271.278 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 421, total running count: 2293 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.272.191 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.286.796 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 422, total running count: 2294 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.287.544 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.302.008 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 423, total running count: 2295 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.302.969 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.317.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 424, total running count: 2296 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.318.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.332.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 425, total running count: 2297 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.333.485 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.348.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 426, total running count: 2298 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.348.987 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.363.586 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 427, total running count: 2299 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.364.149 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.378.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 428, total running count: 2300 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.379.641 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.394.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 429, total running count: 2301 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.394.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.409.437 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 430, total running count: 2302 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.410.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.424.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 431, total running count: 2303 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.425.740 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.440.419 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 432, total running count: 2304 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.441.079 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.455.885 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 433, total running count: 2305 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.459.686 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.474.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 434, total running count: 2306 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.475.620 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.490.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 435, total running count: 2307 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.491.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.505.880 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 436, total running count: 2308 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.506.877 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.521.329 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 437, total running count: 2309 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.522.592 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.537.150 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 438, total running count: 2310 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.537.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.552.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 439, total running count: 2311 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.553.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.568.056 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 440, total running count: 2312 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.569.300 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.583.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 441, total running count: 2313 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.585.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.599.612 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 442, total running count: 2314 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.600.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.615.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 443, total running count: 2315 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.616.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.630.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 444, total running count: 2316 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.631.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.646.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 445, total running count: 2317 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.646.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.661.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 446, total running count: 2318 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.662.226 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.676.779 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 447, total running count: 2319 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.677.752 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.692.301 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 448, total running count: 2320 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.693.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.707.686 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 449, total running count: 2321 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.708.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.723.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 450, total running count: 2322 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.723.822 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.738.624 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 451, total running count: 2323 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.739.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.753.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 452, total running count: 2324 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.757.200 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.771.709 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 453, total running count: 2325 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.772.944 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.787.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 454, total running count: 2326 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.789.558 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.804.129 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 455, total running count: 2327 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.806.115 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.820.808 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 456, total running count: 2328 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.822.148 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.836.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 457, total running count: 2329 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.837.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.852.303 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 458, total running count: 2330 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.853.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.867.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 459, total running count: 2331 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.868.813 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.883.737 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 460, total running count: 2332 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.884.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.899.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 461, total running count: 2333 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.903.764 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.918.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 462, total running count: 2334 [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.920.515 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.935.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 463, total running count: 2335 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.936.459 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.951.207 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 464, total running count: 2336 [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:03.952.330 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.966.925 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 465, total running count: 2337 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:03.967.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.982.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 466, total running count: 2338 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:03.983.798 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:03.998.396 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 467, total running count: 2339 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:04.000.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:04.015.100 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 468, total running count: 2340 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:04.016.422 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_1 execution count: 5, execution time: 7300.54 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:04.016.568 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_1 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.016.643 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty epoch: 5 step: 468, loss is 0.12027578800916672 [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.017.277 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.h:76] ContinueSend] continue send at the beginning of the epoch [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.017.402 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.017.499 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.072.854 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at t10k-images-idx3-ubyte. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.072.928 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at t10k-labels-idx1-ubyte. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.073.084 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.074.808 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.074.926 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.074.948 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.074.965 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.075.001 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.075.072 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.075.103 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.075.121 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.075.143 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.075.162 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.075.196 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.075.233 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.075.258 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.075.292 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.075.648 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at t10k-images-idx3-ubyte. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.075.679 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at t10k-labels-idx1-ubyte. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.224.797 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.224.902 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.236.453 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.236.574 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.236.598 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.236.614 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.236.646 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.236.742 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.236.773 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.236.789 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.236.808 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.236.827 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.236.861 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.236.874 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.236.896 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.236.925 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.237.252 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at t10k-images-idx3-ubyte. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.237.280 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at t10k-labels-idx1-ubyte. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.369.274 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.369.395 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.380.454 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.380.572 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.380.621 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.380.642 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.380.678 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.380.754 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.380.784 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.380.803 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.380.826 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.380.849 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.380.884 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.380.898 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.380.919 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.380.951 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.381.285 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] ME(164043:281473398948928,MainProcess):2024-01-10-11:41:04.382.019 [mindspore/dataset/engine/datasets.py:4269] queue_name is newly generated. value is 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.383.796 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.383.897 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.383.921 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.383.942 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.383.972 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.384.042 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.384.072 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.384.086 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.384.109 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.384.127 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.384.161 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.384.177 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.384.197 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.384.227 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.384.538 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.384.675 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1729] InitExecDatasetVm] Start InitDataSet Entry [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:04.384.839 [mindspore/ccsrc/backend/graph_compiler/transform.cc:573] CreateBackend] CreateBackend is: ge [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:04.384.868 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:04.384.887 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEBUG(164043,ffffa1f54440,python):2024-01-10-11:41:04.384.939 [mindspore/ccsrc/debug/debugger/debugger.cc:129] Init] Debugger got device_id: 2 [INFO] DEBUG(164043,ffffa1f54440,python):2024-01-10-11:41:04.384.956 [mindspore/ccsrc/debug/debugger/debugger.cc:131] Init] Debugger got device_target: Ascend [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:04.385.007 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:04.385.025 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.385.037 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1252] SetRunMode] Run graph mode with kernel by kernel by configuration. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:04.385.068 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:546] CompileGraphs] Status record: start compile function graph: _anonymous__377 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.385.142 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unify_mindir_pm_0_erase_invalid_micro_depend in 4.87 us [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:04.385.231 [mindspore/ccsrc/utils/anfalgo.cc:1736] IsNodeOutputDynamicShape] Invalid base shape, node: Default/Return-op0_6 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:04.385.294 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:04.385.313 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:04.385.352 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:04.385.367 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:04.385.391 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:04.385.409 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:04.385.441 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:727] CompileGraph] Compile graph: _anonymous__377, Split segments size: 2 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:04.385.476 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:762] CompileGraphFromSegment] Compile normal segment, the first node: @_anonymous__377:CNode_378{[0]: ValueNode InitDataSetQueue} [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:04.385.549 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:421] CompileGraph] Status record: start compile graph. [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:04.385.583 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:1940] ConstructKernelGraph] Create graph: 2 [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:04.385.660 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:2697] ConstructOutput] Output:@_anonymous__377:CNode_378{[0]: ValueNode InitDataSetQueue} [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.385.948 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:193] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:2 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.386.018 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 7.74 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.386.107 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 57.69 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.386.222 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:154] CommonUnifyMindIR] start common unify mindir opt graph:2 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.386.269 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.386.347 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 73.05 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.386.369 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.386.405 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 31.54 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.386.424 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.386.442 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 17.9 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.386.455 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.386.503 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 43.91 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.386.622 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:79] BackendCommonOptimization] Status record: start common optimization. graph id: 2 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.386.668 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: add_dynamic_shape_attr [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.386.717 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_0_add_dynamic_shape_attr in 43.64 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.386.733 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.386.786 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_1_convert_dynamic_broadcast_to in 48.26 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.386.832 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_2_convert_const_input_to_attr in 26.97 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.386.875 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_3_custom_op_const_input_to_attr in 17.1 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.386.909 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 14.5 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.386.990 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 59.78 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.015 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 0.95 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.059 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_7_flatten_concat_fission in 26.29 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.098 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 19.15 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.137 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_9_broadcast_to_fusion in 18.29 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.176 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 18.66 us [WARNING] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.281 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:92] BackendCommonOptimization] [PROF]backend_common_optimization costs 659 usec. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.302 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:99] BackendCommonOptimization] Status record: end common optimization. graph id: 2 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.463 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_0_renorm_split in 31.28 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.489 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.564 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_1_reduce_axis_update in 69.11 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.587 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim HistogramFixedWidth not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.604 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_2_histogram_fixed_width_fusion in 15.01 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.619 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim ClipByNorm not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.634 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_3_clip_by_norm_fission in 13.95 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.649 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.674 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_4_tensor_arry_add_flow_cond1 in 24.98 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.691 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayGather not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.705 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_5_tensor_arry_add_flow_cond2 in 12.08 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.717 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.731 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_6_ge_tensor_arry_cast_index in 11.49 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.743 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArray not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.759 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_7_tensor_array_prepare in 14.65 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.772 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.806 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_8_space_to_batch_nd_attr_update in 31.04 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.827 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.857 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_9_batch_to_space_nd_attr_update in 27.05 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.873 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: avg_pool_grad_unify_mindir [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.914 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_10_avg_pool_grad_unify_mindir in 37.51 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.387.966 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_11_adam_weight_decay_unify_mindir in 28.65 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.013 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_12_cdist_fission in 22.37 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.060 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_13_cdist_grad_fission in 22.21 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.132 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_14_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 49.59 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.192 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_15_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 34.53 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.237 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_16_sparse_softmax_cross_entropy_with_logits_unify_mindir in 21.6 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.280 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_17_dropout_unify_mindir1 in 18.5 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.310 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.357 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_18_dropoutgrad_unify_mindir in 41.84 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.378 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.393 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_19_neighbor_exchange_unify_mindir in 16.13 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.411 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.424 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_20_neighbor_exchange_v2_unify_mindir in 12.14 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.437 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.450 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_21_neighbor_exchange_v2_grad_unify_mindir in 11.61 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.464 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.480 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_22_all_to_all_unify_mindir in 15 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.497 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim QuantDTypeCast not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.516 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_23_quant_dtype_cst_adjust in 18.19 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.530 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim FSEDecode not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.548 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_24_fse_decode_adjust in 17.92 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.567 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.583 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_25_bn_split in 15.52 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.599 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.657 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_26_bn_grad_unify_mindir in 54.42 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.677 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.695 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_27_bn_grad_split in 16.09 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.721 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.736 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_28_batchnorm_to_bninfer in 13.08 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.752 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.769 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_29_batchnormgrad_to_bninfergrad in 14.61 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.781 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.794 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_30_batch_norm_grad_infer_fission in 11.18 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.844 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_31_ascend_mindir_op_adapter in 29.83 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.868 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_32_DropoutGenMask in 1.05 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.913 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_33_lamb_fission_ge in 24.53 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.388.976 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_34_print_insert_placeholder_for_tensor_name in 37.17 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.389.023 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_35_getnext_for_ge in 22.34 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.389.069 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_36_sync_bn_split in 21.11 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.389.112 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_37_sync_bn_grad_split in 19.62 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.389.153 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_38_adaptive_max_pool2d_ge_fusion in 20.83 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.389.194 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_39_avg_pool_grad_for_ge in 17.3 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.389.213 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:260] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.389.320 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_40_FlashAttentionFusionV1 in 104.4 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.389.344 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:374] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.389.457 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_41_FlashAttentionFusionV1 in 109.89 us [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:04.389.874 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:04.389.918 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:04.389.932 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:132] GetRunMode] RunMode::kKernelMode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.390.038 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 66.23 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.390.314 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_0_seed_adapter in 58.03 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.390.344 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: env_op_attr_update [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.390.386 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_1_env_op_attr_update in 37.5 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.390.433 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_2_process call inline in 24.98 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.390.463 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_3_expander_fallback in 10.43 us [WARNING] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.390.542 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:139] GEBackendOptimizeACL] [PROF]ascend_backend_optimize_acl costs 304 usec. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:04.390.578 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] InitDataSetQueue is not defined in opdef. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.390.728 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 8.02 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.390.800 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 46.24 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.390.860 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 35.41 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.390.936 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 52.85 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.390.978 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_4_insert_type_transform_op in 18.68 us [WARNING] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.063 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:179] GEBackendOptimizeACLAfterKernelSelect] [PROF]ascend_backend_optimize_acl_after_kernel_select costs 388 usec. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.123 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_0_DropoutGenMask in 0.55 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.171 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_1_cse in 25.34 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.226 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_redundant_op in 35.81 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.254 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_3_eliminate_maketuple_getitem in 7.15 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.284 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_4_MergeTransData in 0.88 us [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.412 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive InitDataSetQueue [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.484 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive InitDataSetQueue [WARNING] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.503 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:262] CreateKernel] [PROF]create_kernel costs 100 usec. [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.542 [mindspore/ccsrc/backend/common/session/session_basic.cc:1140] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 2 start [INFO] DEBUG(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.558 [mindspore/ccsrc/debug/summary/summary.cc:33] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 2 start [INFO] DEBUG(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.573 [mindspore/ccsrc/debug/summary/summary.cc:54] RecurseSetSummaryNodesForAllGraphs] The total summary nodes is: 0 for graph: 2 [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.615 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:891] PrintGraphExecuteOrder] Graph 2 execution order: [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.659 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[0], node name[Default/InitDataSetQueue-op0_6], logic id[4294967295], stream id[0], node info[@kernel_graph_2:CNode_378{[0]: ValueNode InitDataSetQueue}] [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.688 [mindspore/ccsrc/backend/common/somas/somas.cc:143] Assign] Start Somas Assign for graph 2 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.719 [mindspore/ccsrc/backend/common/somas/somas.cc:151] Assign] Somas Configure success, configuration info: Device Name: Ascend Run by execution order: 1 Enable debug log: 0 Debug log path: . [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.730 [mindspore/ccsrc/backend/common/somas/somas.cc:154] Assign] Start Initialize SOMAS Model [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.772 [mindspore/ccsrc/backend/common/somas/somas.cc:1065] GraphOutputProcess] Set 0 graph output tensors' aligned size to 0. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.810 [mindspore/ccsrc/backend/common/somas/somas.cc:1135] RefNodeProcess] Special Tensor total size: RefNode: input 0 output 0 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.833 [mindspore/ccsrc/backend/common/somas/somas.cc:551] InitSomasModel] No Tensor from graph 2 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.845 [mindspore/ccsrc/backend/common/somas/somas.cc:157] Assign] End Initialize SOMAS Model [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.856 [mindspore/ccsrc/backend/common/somas/somas.cc:160] Assign] No Somas Tensor in graph 2 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.867 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:310] DoSomas] Somas allocate success for graph 2 somas size: 0 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.887 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:801] CreateDeviceAddress] Status record: start create device address. graph id: 2 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.943 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:808] CreateDeviceAddress] Status record: end create device address. graph id: 2 [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:04.391.985 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1129] CacheGraphOutputToFrontNodeWithIndex] Get graph backend output nodes. [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:04.392.006 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1137] CacheGraphOutputToFrontNodeWithIndex] Get graph front output nodes. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:04.392.024 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:507] CompileGraph] Status record: end compile graph. graph id: 2 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:04.392.099 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:805] CompileGraphFromSegment] Compile cut segment, the cut node: @_anonymous__377:CNode_379{[0]: ValueNode Return, [1]: CNode_378} [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:04.392.162 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:521] Transform] Graph(kernel_graph_2) transforms actor begin, strategy:pipeline_with_execution_order [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:04.392.228 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1232] BuildLoopCountActor] Create loop count actor: kernel_graph_2_LoopCountActor [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:04.392.248 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1265] BuildOutputActor] Create output actor: kernel_graph_2_OutputActor [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:04.392.266 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1285] BuildDataPrepareActor] Create data prepare actor: kernel_graph_2_DataPrepareActor [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:04.392.296 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1518] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_2 start. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:04.392.313 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1520] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_2 end. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:04.392.394 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 1_actor_set_kernel_graph_2_memory_actor_insert in 1.34 us [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:04.392.416 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 2_actor_set_kernel_graph_2_invalid_data_arrow_elimination in 1.17 us [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:04.392.447 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 3_actor_set_kernel_graph_2_multi_actor_fusion in 16.03 us [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:04.392.463 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 4_actor_set_kernel_graph_2_batch_data_arrow_fusion in 0.82 us [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:04.392.479 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:554] Transform] Graph(kernel_graph_2) transforms actor end. [WARNING] VM(164043,ffffa1f54440,python):2024-01-10-11:41:04.392.533 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:612] CompileGraphs] [PROF]compile_func_graph costs 7446 usec. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:04.392.555 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:615] CompileGraphs] Status record: end compile function graph: _anonymous__377, produce actor: kernel_graph_2 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:04.392.574 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_2 [INFO] GE(164043,python):2024-01-10-11:41:04.466.675 [graph_var_manager.cc:1424][EVENT]167073 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:41:04.466.771 [graph_manager.cc:1248][EVENT]167073 PreRun:PreRun start: graph node size 1, session id 31, graph id 30, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:41:04.467.046 [atrace_api.c:28](tid:167073) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:41:04.467.074 [trace_rb_log.c:84](tid:167073) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:41:04.467.087 [atrace_api.c:32](tid:167073) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:41:04.467.104 [client_manager.cpp:157][SetProfilingCallback][tid:167073] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:41:04.467.465 [parallel_partitioner.cc:165][EVENT]167073 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.467.499 [parallel_partitioner.cc:178][EVENT]167073 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.467.542 [graph_prepare.cc:1378][EVENT]167073 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.467.668 [graph_manager.cc:1050][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [140] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.467.689 [graph_manager.cc:1052][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.467.747 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [1] [INFO] GE(164043,python):2024-01-10-11:41:04.467.774 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.467.826 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [40] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.467.839 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.467.896 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.467.909 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.467.920 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.468.030 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [3] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.468.050 [graph_manager.cc:1054][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [349] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.468.279 [graph_manager.cc:1055][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [216] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.468.751 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:41:04.468.774 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.468.811 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.468.821 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferShapePass is [99] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.468.830 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.468.839 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:41:04.468.848 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.468.856 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.468.864 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.033 [graph_manager.cc:1056][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [1735] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.470.093 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.111 [graph_prepare.cc:1982][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [44] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.470.266 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:41:04.470.283 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [0] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.293 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.303 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferShapePass is [49] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.311 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [4] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.320 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:41:04.470.328 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [0] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.337 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.345 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of InferValuePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.370 [graph_prepare.cc:1983][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [245] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.470.391 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.470.402 [graph_prepare.cc:1984][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.470.416 [graph_prepare.cc:1985][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.470.440 [graph_prepare.cc:1986][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.470.451 [graph_prepare.cc:1987][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.470.465 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.470.476 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.470.490 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.470.557 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.569 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.579 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.587 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [0] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.596 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.604 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.613 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.621 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.630 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.638 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.646 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.655 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.663 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.672 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [3] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.680 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.688 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.470.708 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.470.722 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.470.755 [graph_prepare.cc:1988][EVENT]167073 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [295] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.470.768 [graph_manager.cc:1065][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [700] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.483.582 [graph_manager.cc:1077][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12795] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.483.633 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.483.661 [graph_manager.cc:1080][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [39] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.277 [graph_manager.cc:1081][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [2598] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.324 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.340 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.354 [graph_manager.cc:1082][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [39] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.387 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.403 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.421 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.455 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [22] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.474 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.491 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.507 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.539 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.556 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.576 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.596 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.611 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.637 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.649 [graph_manager.cc:2700][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [266] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.726 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.486.743 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.486.752 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.486.761 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.486.769 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.486.778 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CastRemovePass is [7] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.486.786 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.486.795 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.486.807 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.486.815 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.486.824 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [4] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.486.832 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [0] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.486.841 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.486.849 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.486.857 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.486.867 [graph_manager.cc:2741][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [200] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.877 [graph_manager.cc:2752][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.899 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.910 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.925 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.940 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.959 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.971 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.486.990 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.008 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.022 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.033 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.047 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.058 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.071 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.083 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.093 [graph_manager.cc:2810][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [198] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.116 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.487.127 [graph_manager.cc:2821][EVENT]167073 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.154 [graph_manager.cc:1087][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [779] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.285 [graph_manager.cc:1088][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [118] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.321 [graph_manager.cc:1089][EVENT]167073 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.339 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.352 [graph_manager.cc:1097][EVENT]167073 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:41:04.487.372 [graph_manager.cc:3325][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.476 [engine_place.cc:144][EVENT]167073 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.492 [engine_place.cc:144][EVENT]167073 Run:The time cost of aicpu_ascend_kernel::CheckSupported is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.552 [graph_manager.cc:3351][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [167] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.580 [graph_manager.cc:3364][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.640 [engine_partitioner.cc:1139][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.660 [engine_partitioner.cc:1142][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.768 [engine_partitioner.cc:1148][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [95] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.797 [engine_partitioner.cc:1155][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.835 [engine_partitioner.cc:1164][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [27] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.866 [graph_manager.cc:3405][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [274] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.487.883 [graph_manager.cc:3412][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.489.369 [graph_manager.cc:3422][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [1472] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.489.404 [graph_manager.cc:3428][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.489.515 [graph_manager.cc:3467][EVENT]167073 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [90] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.489.532 [graph_manager.cc:3377][EVENT]167073 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [1940] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.489.547 [graph_manager.cc:1106][EVENT]167073 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [2181] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.489.559 [graph_manager.cc:1115][EVENT]167073 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:41:04.489.580 [graph_manager.cc:1130][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.489.611 [graph_manager.cc:1131][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.489.634 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.489.651 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.489.660 [graph_manager.cc:2837][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [34] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.489.774 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [69] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.489.797 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.489.807 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of CondRemovePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.489.815 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.489.824 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.489.833 [base_pass.cc:339][EVENT]167073 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:04.489.843 [graph_manager.cc:2864][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [167] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.489.854 [graph_manager.cc:2872][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.489.873 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.489.886 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.489.900 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.489.913 [compile_nodes_pass.cc:88][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.489.927 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.489.942 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.489.971 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.489.997 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.008 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.021 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.034 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.044 [graph_manager.cc:2927][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [173] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.059 [graph_manager.cc:2937][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.083 [graph_manager.cc:2943][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.103 [graph_manager.cc:2950][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.296 [graph_manager.cc:2958][EVENT]167073 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [27] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.323 [graph_manager.cc:1132][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [698] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.429 [graph_manager.cc:1135][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [93] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.459 [graph_manager.cc:2975][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.520 [graph_manager.cc:2981][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [47] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.534 [pass_manager.cc:82][EVENT]167073 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.545 [graph_manager.cc:2986][EVENT]167073 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.554 [graph_manager.cc:1136][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [109] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.628 [graph_manager.cc:3555][EVENT]167073 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [49] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.675 [engine_partitioner.cc:1139][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.689 [engine_partitioner.cc:1142][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.754 [engine_partitioner.cc:1148][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [56] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.775 [engine_partitioner.cc:1155][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.805 [engine_partitioner.cc:1164][EVENT]167073 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.826 [graph_builder.cc:865][EVENT]167073 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [172] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.887 [graph_builder.cc:288][EVENT]167073 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [44] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.490.975 [graph_builder.cc:293][EVENT]167073 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [74] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.491.154 [model_builder.cc:1133][EVENT]167073 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [94] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.491.344 [block_mem_assigner.cc:4069][EVENT]171958 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164043,python):2024-01-10-11:41:04.491.344 [block_mem_assigner.cc:4069][EVENT]171957 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164043,python):2024-01-10-11:41:04.491.620 [model_builder.cc:1144][EVENT]167073 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [445] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.491.647 [model_builder.cc:1152][EVENT]167073 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.491.662 [model_builder.cc:1157][EVENT]167073 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.491.769 [model_builder.cc:1167][EVENT]167073 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [96] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.491.787 [model_builder.cc:1174][EVENT]167073 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.491.807 [model_builder.cc:1180][EVENT]167073 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.491.843 [model_builder.cc:1184][EVENT]167073 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [25] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.491.862 [graph_builder.cc:304][EVENT]167073 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [867] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:41:04.491.957 [logger.cc:1071] 167073 ModelBindStream: model_id=576, stream_id=1857, flag=0. [INFO] GE(164043,python):2024-01-10-11:41:04.492.019 [task_generator.cc:804][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.492.070 [task_generator.cc:805][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.492.674 [task_generator.cc:814][EVENT]167073 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [587] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.492.692 [task_generator.cc:954][EVENT]167073 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [679] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.492.745 [task_generator.cc:967][EVENT]167073 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [29] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:41:04.492.764 [logger.cc:1084] 167073 ModelUnbindStream: model_id=576, stream_id=1857, [INFO] GE(164043,python):2024-01-10-11:41:04.492.821 [graph_builder.cc:310][EVENT]167073 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [947] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.492.925 [graph_manager.cc:1152][EVENT]167073 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2354] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.492.942 [graph_manager.cc:1164][EVENT]167073 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:41:04.492.973 [graph_manager.cc:1271][EVENT]167073 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [25583] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.492.983 [graph_manager.cc:1272][EVENT]167073 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:41:04.493.292 [atrace_api.c:93](tid:167073) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:41:04.493.312 [atrace_api.c:95](tid:167073) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:41:04.493.944 [model_introduction.cc:236][EVENT]167073 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(164043,python):2024-01-10-11:41:04.493.967 [model_introduction.cc:294][EVENT]167073 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(164043,python):2024-01-10-11:41:04.493.990 [model_introduction.cc:366][EVENT]167073 Data:model io_info size:0 [INFO] GE(164043,python):2024-01-10-11:41:04.495.982 [graph_converter.cc:838][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [661] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.496.050 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.496.261 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [193] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.496.323 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [43] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.496.336 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [57] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.496.359 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.496.383 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.496.402 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of ZeroCopy is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.496.435 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CEM is [24] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.496.478 [copy_flow_launch_fuse.cc:395][EVENT]167073 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [32] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.496.488 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [42] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.496.505 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.496.523 [base_optimizer.cc:70][EVENT]167073 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.496.535 [graph_converter.cc:849][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [509] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.496.655 [graph_converter.cc:853][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [111] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.497.046 [graph_converter.cc:857][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [378] micro second. [INFO] GE(164043,python):2024-01-10-11:41:04.497.115 [graph_converter.cc:862][EVENT]167073 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [50] micro second. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:04.498.350 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_2_LoopCountActor) running, loop count: 1, current count: 1, total running count: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:04.498.614 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_2 execution count: 1, execution time: 105.944 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:04.498.689 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_2 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:04.498.834 [mindspore/ccsrc/runtime/device/kernel_runtime_manager.cc:35] ClearGraphResource] Clear device Ascend_2 graph 2 runtime resource [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.501.111 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:198] Compile] Input plan: +-Transfer,send_epoch_end:false,total_batch:0) | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.501.237 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:216] Compile] Plan before optimization: +-Top | +-Transfer,send_epoch_end:false,total_batch:0) | | +-Repeat(count:1) | | | +-Batch(batch_size:32 drop_remainder:true) | | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | | +-MnistDataset [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.501.260 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:60] PrePass] Running pre pass loops. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.501.277 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.501.312 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.501.397 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.501.426 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.501.442 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.501.462 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.501.474 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/cache_transform_pass.cc:182] RunOnTree] Pre pass: Cache transform pass started. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.501.494 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/cache_transform_pass.cc:199] RunOnTree] Pre pass: Cache transform pass complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.501.506 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:91] PrePass] Pre pass offload complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.501.518 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:116] PostPass] Running post pass loops. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.501.547 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:135] PostPass] Post passes complete. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:04.501.581 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:230] Compile] Plan after optimization: +-Top | +-Transfer,send_epoch_end:false,total_batch:0) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:04.502.289 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_data_queue.cc:227] AscendTdtQueue] Select MBUF channel, the capacity of data queue is: 128 [INFO] MD(164043,fffd1a7fc0f0,python):2024-01-10-11:41:04.504.566 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at t10k-images-idx3-ubyte. [INFO] MD(164043,fffd1a7fc0f0,python):2024-01-10-11:41:04.504.602 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at t10k-labels-idx1-ubyte. [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.512.198 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:456] SendDataToAscend] Device queue, sending data to Ascend. [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.670.347 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:502] SendDataToAscend] Begin to send data to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.670.448 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:1182] PrintBeginInfoWhenFirstBatch] Loading dataset and begin to push first batch into device ... [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.670.912 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:1192] PrintEndInfoWhenFirstBatch] Loading dataset and push first batch into device successful. [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.670.939 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.671.385 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.671.904 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 3 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.672.375 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 4 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.672.832 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 5 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.673.335 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 6 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.673.782 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 7 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.674.230 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 8 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.674.673 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 9 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.675.124 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 10 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.675.520 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 11 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.675.946 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 12 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.676.380 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 13 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.676.798 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 14 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.677.264 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 15 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.677.682 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 16 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.678.166 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 17 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.678.612 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 18 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.679.079 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 19 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.679.531 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 20 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.679.975 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 21 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.680.396 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 22 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.680.816 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 23 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.681.242 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 24 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.681.643 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 25 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.682.059 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 26 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.682.504 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 27 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.682.914 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 28 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.683.337 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 29 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.683.761 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 30 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.684.213 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 31 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.684.629 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 32 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.685.145 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 33 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.685.595 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 34 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.686.014 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 35 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.686.464 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 36 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.686.926 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 37 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.687.356 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 38 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.687.761 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 39 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.688.171 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 40 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.688.594 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 41 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.689.190 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 42 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.689.628 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 43 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.690.109 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 44 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.690.517 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 45 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.690.929 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 46 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.691.359 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 47 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.691.795 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 48 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.692.194 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 49 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.692.622 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 50 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.693.041 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 51 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.693.555 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 52 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.693.949 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 53 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.694.379 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 54 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.694.788 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 55 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.695.205 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 56 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.695.612 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 57 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.696.039 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 58 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.696.458 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 59 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.696.856 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 60 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.697.397 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 61 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.697.884 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 62 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.698.307 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 63 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.698.728 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 64 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.699.145 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 65 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.699.548 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 66 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.699.945 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 67 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.700.346 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 68 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.700.736 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 69 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.701.268 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 70 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.701.656 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 71 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.702.103 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 72 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.702.490 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 73 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.702.891 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 74 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.703.285 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 75 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.703.677 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 76 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.704.071 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 77 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.704.485 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 78 batch(es) to device, channel name: 14b44e48-af6a-11ee-9398-30fd658829ca [INFO] MD(164043,fffd1bfff0f0,python):2024-01-10-11:41:04.704.653 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:596] SendDataToAscend] ExecutionTree finished. Device queue sent number of batches: 78 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.871.488 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:978] CompileInner] Start compiling, phase: eval.1704858064714363904.281470244928208.0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.871.572 [mindspore/ccsrc/pipeline/jit/ps/event_message_print.cc:37] PrintEventMessage] Start compiling '_DataWrapper.construct' and it will take a while. Please wait... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.871.640 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1659] VmPipeline] This worker is initialized. No need to add worker action. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:04.871.673 [mindspore/ccsrc/backend/graph_compiler/transform.cc:573] CreateBackend] CreateBackend is: ge [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:04.871.693 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:04.871.706 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEBUG(164043,ffffa1f54440,python):2024-01-10-11:41:04.871.930 [mindspore/ccsrc/debug/debugger/debugger.cc:129] Init] Debugger got device_id: 2 [INFO] DEBUG(164043,ffffa1f54440,python):2024-01-10-11:41:04.871.946 [mindspore/ccsrc/debug/debugger/debugger.cc:131] Init] Debugger got device_target: Ascend [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.871.966 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1311] Run] Pipeline run [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.871.987 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start parse action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.873.879 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end parse action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.873.940 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start symbol_resolve action. [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.882.457 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380] Added global python symbol: {_check_is_tensor : Prim[_check_is_tensor]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.883.015 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_381{[0]: CNode_382, [1]: ValueNode logits, [2]: param_logits, [3]: CNode_383}, block: 0x263b0950/mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/nn/loss/loss.py:777/ _check_is_tensor('logits', logits, self.cls_name)/ [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.883.566 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_384{[0]: CNode_382, [1]: ValueNode labels, [2]: param_labels, [3]: CNode_385}, block: 0x263b0950/mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/nn/loss/loss.py:778/ _check_is_tensor('labels', labels, self.cls_name)/ [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.884.282 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_386{[0]: ValueNode Depend, [1]: CNode_387, [2]: CNode_388}, state: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_389{[0]: ValueNode MakeTuple, [1]: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_381{[0]: CNode_382, [1]: ValueNode logits, [2]: param_logits, [3]: CNode_383}, [2]: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_384{[0]: CNode_382, [1]: ValueNode labels, [2]: param_labels, [3]: CNode_385}} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.886.595 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_391, [1]: param_x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.886.890 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_392, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.887.172 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_393, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.887.455 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_394, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.887.728 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_395, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.887.996 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_396, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.888.270 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_397, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.888.543 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_398, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.888.811 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_399, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.889.081 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_400, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.889.346 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_401, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.889.621 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_402, [1]: x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.892.262 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_403] Added global python symbol: {len : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.892.436 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.892.794 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.892.973 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.893.462 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_403:CNode_405{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.893.605 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_404:x{[0]: CNode_406, [1]: param_фx, [2]: CNode_405} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.894.120 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_403:CNode_407{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.894.586 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_404] Added global python symbol: {len : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.894.657 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_403:CNode_408{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.894.768 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_403:x_shape{[0]: CNode_409, [1]: param_x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.894.907 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.895.204 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.895.489 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_404] Added global python symbol: {F : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.895.586 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_403] Added global python symbol: {F : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.895.645 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_403:CNode_410{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.895.986 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.900.516 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_411] Added global python symbol: {len : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.900.682 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.901.027 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.901.205 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.901.671 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_411:CNode_413{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.901.857 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_412:x{[0]: CNode_414, [1]: param_фx, [2]: CNode_413} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.902.302 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_411:CNode_415{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.902.755 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_412] Added global python symbol: {len : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.902.825 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_411:CNode_416{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.902.934 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_411:x_shape{[0]: CNode_417, [1]: param_x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.903.071 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.903.351 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.903.620 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_412] Added global python symbol: {F : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.903.715 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_411] Added global python symbol: {F : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.903.792 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_411:CNode_418{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.904.109 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.908.228 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_419] Added global python symbol: {len : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.908.404 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.908.747 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.908.932 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.909.400 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_419:CNode_421{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.909.540 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_420:x{[0]: CNode_422, [1]: param_фx, [2]: CNode_421} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.910.050 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_419:CNode_423{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.910.507 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_420] Added global python symbol: {len : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.910.576 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_419:CNode_424{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.910.687 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_419:x_shape{[0]: CNode_425, [1]: param_x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.910.826 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.911.104 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.911.399 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_420] Added global python symbol: {F : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.911.492 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_419] Added global python symbol: {F : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.911.546 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_419:CNode_426{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.911.861 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.914.020 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Flatten_construct_427] Added global python symbol: {F : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.916.946 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1261] ParseNameConstant] The NameConstant is bool:False [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.917.240 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:3 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.917.502 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.917.635 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1261] ParseNameConstant] The NameConstant is bool:True [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.918.404 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_pooling_MaxPool2d_construct_428] Added global python symbol: {isinstance : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.918.513 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_pooling_MaxPool2d_construct_429] Added global python symbol: {isinstance : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.918.575 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_pooling_MaxPool2d_construct_428 update var `isinstance` with node @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_430{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.pooling', [2]: ValueNode isinstance} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.918.743 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_pooling_MaxPool2d_construct_428] Added global python symbol: {tuple : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.918.830 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_pooling_MaxPool2d_construct_429] Added global python symbol: {tuple : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.918.882 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_pooling_MaxPool2d_construct_428 update var `tuple` with node @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_431{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.pooling', [2]: ValueNode tuple} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.919.249 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.919.371 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.919.576 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.919.688 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.919.970 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.931.805 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.931.972 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.932.542 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @canonicalize_axis_432:CNode_433{[0]: ValueNode check_axis_valid_434, [1]: param_axis, [2]: ndim}, block: 0x2b2fa320/canonicalize_axis_432, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1606/ check_axis_valid(axis, ndim)/ [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.932.709 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.933.009 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @canonicalize_axis_432:CNode_435{[0]: ValueNode Depend, [1]: CNode_436, [2]: CNode_437}, state: @canonicalize_axis_432:CNode_433{[0]: ValueNode check_axis_valid_434, [1]: @canonicalize_axis_432:param_axis, [2]: @canonicalize_axis_432:ndim{[0]: CNode_438}} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.933.301 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {isinstance : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.933.457 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {Tensor : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.934.068 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {int : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.934.553 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {bool : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.935.255 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {check_flatten_order_const : Prim[check_flatten_order]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.935.750 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @2↓flatten_440:CNode_441{[0]: CNode_442, [1]: param_order}, block: 0x28576240/2↓flatten_440, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1615/ check_flatten_order_const(order)/ [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.936.180 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {rank_ : Prim[Rank]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.936.563 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.936.631 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.936.862 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {reshape_ : Prim[Reshape]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.937.052 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.937.358 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {ops : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.937.562 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.938.122 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {transpose_ : Prim[Transpose]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.938.544 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_443] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.938.653 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.938.722 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `shape_` with node @flatten_439:CNode_444{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode shape_} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.939.062 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_443] Added global python symbol: {rank_ : Prim[Rank]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.939.137 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `rank_` with node @flatten_439:CNode_445{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode rank_} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.939.448 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `start_dim` with node @flatten_439:param_start_dim [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.939.615 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.939.767 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `end_dim` with node @flatten_439:param_end_dim [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.939.886 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.940.160 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.940.215 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.940.442 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_443] Added global python symbol: {reshape_ : Prim[Reshape]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.940.533 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `reshape_` with node @flatten_439:CNode_446{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode reshape_} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.940.732 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.941.047 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_443] Added global python symbol: {flatten_ : Prim[Flatten]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.941.158 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {flatten_ : Prim[Flatten]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.941.230 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `flatten_` with node @flatten_439:CNode_447{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode flatten_} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.941.564 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `canonicalize_axis` with node ValueNode canonicalize_axis_432 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.942.029 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `check_dim_valid` with node ValueNode check_dim_valid_448 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.942.507 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @4↓flatten_449:CNode_450{[0]: ValueNode check_dim_valid_448, [1]: start_dim, [2]: end_dim}, block: 0x285fb760/4↓flatten_449, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1636/ check_dim_valid(start_dim, end_dim)/ [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.942.755 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.942.813 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.943.092 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.943.569 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.944.129 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.944.750 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.945.211 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @2↓flatten_440:CNode_451{[0]: ValueNode Depend, [1]: CNode_452, [2]: CNode_453}, state: @2↓flatten_440:CNode_441{[0]: @flatten_439:CNode_442{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode check_flatten_order_const}, [1]: @flatten_439:param_order} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.945.332 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @4↓flatten_449:CNode_454{[0]: ValueNode Depend, [1]: CNode_455, [2]: CNode_456}, state: @4↓flatten_449:CNode_450{[0]: ValueNode check_dim_valid_448, [1]: @4↓flatten_449:idx{[0]: ValueNode canonicalize_axis_432, [1]: param_start_dim, [2]: x_rank}, [2]: @4↓flatten_449:end_dim{[0]: ValueNode canonicalize_axis_432, [1]: param_end_dim, [2]: x_rank}} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.945.459 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.945.552 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.946.790 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:174] CheckFuncReturn] Function must has 'return' statement, but missing in function 'flatten' at /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1557. FuncGraph: ↓check_dim_valid_457. We will add a 'return None' statement automatically. [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.946.969 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:174] CheckFuncReturn] Function must has 'return' statement, but missing in function 'flatten' at /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1557. FuncGraph: ↓check_axis_valid_458. We will add a 'return None' statement automatically. [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.959.552 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [shape_459] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.967.608 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end symbol_resolve action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.967.663 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start graph_reusing action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.967.681 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.basic.DenseDense[True, None]_ID [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.967.697 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.conv.Conv2dConv2dodict_values([6, 16, 5, 1, 'valid', 0, False, ])_ID [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.967.708 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.conv.Conv2dConv2dodict_values([1, 6, 5, 1, 'valid', 0, False, ])_ID [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.967.723 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end graph_reusing action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.967.742 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start meta_unpack_prepare action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.968.604 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end meta_unpack_prepare action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.968.644 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pre_cconv action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.968.662 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pre_cconv action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:04.968.682 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start abstract_specialize action. [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:04.970.254 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_463{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:04.970.318 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:04.970.719 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_464{[0]: CNode_465}, new_node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_466{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:04.970.775 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_464{[0]: CNode_465}, new node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_464{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.971.959 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_equal_scalar_467] Added global python symbol: {F : } [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:04.972.336 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 4, Shape: NoShape), AbstractScalar(Type: Int64, Value: 3, Shape: NoShape)}, g: _equal_scalar_467 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:04.973.001 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_468:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_468:CNode_470{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:04.973.070 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_468:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_468:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:04.975.466 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_472{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:04.975.529 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:04.975.854 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_473{[0]: CNode_474}, new_node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_475{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:04.975.920 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_473{[0]: CNode_474}, new node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_473{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:04.976.572 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_476:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_476:CNode_477{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:04.976.640 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_476:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_476:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.982.270 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_logical_not_scala_478] Added global python symbol: {auto_generate : } [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:04.982.715 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Bool, Value: true, Shape: NoShape)}, g: _logical_not_scala_478 [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.984.346 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bool_not_479] Added global python symbol: {_get_cache_prim : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.984.488 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bool_not_479] Added global python symbol: {BoolNot : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.988.107 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_480] Added global python symbol: {str : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.988.552 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↻_get_cache_prim_for_pynative_481] Added global python symbol: {str : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.988.822 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↻_get_cache_prim_for_pynative_481 update var `str` with node @↵_get_cache_prim_for_pynative_482:param_фstr [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.989.057 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_480] Added global python symbol: {tuple : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.989.273 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] _get_cache_prim_for_pynative_483 update var `key` with node @_get_cache_prim_for_pynative_483:key{[0]: CNode_484, [1]: key, [2]: CNode_485} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.990.070 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_486] Added global python symbol: {str : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.990.698 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_486] Added global python symbol: {_PRIM_CACHE : {}} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.990.793 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_480] Added global python symbol: {_PRIM_CACHE : {}} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.991.047 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_486] Added global python symbol: {Primitive : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.991.139 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_480] Added global python symbol: {Primitive : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.991.838 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @✓↓_get_cache_prim_for_pynative_487:CNode_488{[0]: ValueNode MetaFuncGraph-unpack_call.489, [1]: CNode_490, [2]: param_фargs, [3]: param_фkwargs}, block: 0x2b17c210/✓↓_get_cache_prim_for_pynative_487, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/_primitive_cache.py:84/ prim.__init__(*args, **kwargs)/ [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.992.444 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 2↓_get_cache_prim_for_pynative_491 update var `key` with node @↓_get_cache_prim_for_pynative_492:key{[0]: param_фstr, [1]: param_фkey} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.992.606 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @↻_get_cache_prim_for_pynative_493:CNode_494{[0]: ValueNode Depend, [1]: CNode_495, [2]: CNode_496}, state: @↻_get_cache_prim_for_pynative_493:CNode_497{[0]: ValueNode MetaFuncGraph-add.144, [1]: @↵_get_cache_prim_for_pynative_486:param_@CNode_497, [2]: ValueNode 1} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.992.708 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @✓↓_get_cache_prim_for_pynative_487:CNode_498{[0]: ValueNode Depend, [1]: CNode_499, [2]: CNode_500}, state: @✓↓_get_cache_prim_for_pynative_487:CNode_488{[0]: ValueNode MetaFuncGraph-unpack_call.489, [1]: @✓↓_get_cache_prim_for_pynative_487:CNode_490{[0]: ValueNode getattr, [1]: prim, [2]: ValueNode __init__}, [2]: @↵_get_cache_prim_for_pynative_486:param_фargs, [3]: @↵_get_cache_prim_for_pynative_486:param_фkwargs} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:04.994.107 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_501:CNode_502{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:04.994.177 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_501:CNode_503{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:04.994.221 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_501:CNode_504{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:04.994.858 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BoolNot. node: @bool_not_479:CNode_505{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x}, new_node: @bool_not_479:CNode_506{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:04.994.929 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BoolNot. node: @bool_not_479:CNode_505{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x}, new node: @bool_not_479:CNode_505{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:04.999.211 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_equal_string_507] Added global python symbol: {F : } [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:04.999.558 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: String, Value: C, Shape: NoShape), AbstractScalar(Type: String, Value: F, Shape: NoShape)}, g: _equal_string_507 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.001.075 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_508:CNode_509{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_508:CNode_510{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.001.145 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_508:CNode_509{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_508:CNode_509{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:05.002.428 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_neg_scalar_511] Added global python symbol: {F : } [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.002.752 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 1, Shape: NoShape)}, g: _neg_scalar_511 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.003.364 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarUsub. node: @_neg_scalar_512:CNode_513{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x}, new_node: @_neg_scalar_512:CNode_514{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.003.429 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarUsub. node: @_neg_scalar_512:CNode_513{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x}, new node: @_neg_scalar_512:CNode_513{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.004.097 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_515:CNode_516{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_515:CNode_517{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.004.166 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_515:CNode_516{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_515:CNode_516{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.004.621 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @3↓flatten_518:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput}, new_node: @3↓flatten_518:CNode_519{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.004.685 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @3↓flatten_518:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput}, new node: @3↓flatten_518:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:05.006.115 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_520] Added global python symbol: {F : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:05.006.793 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_520] Added global python symbol: {InSequence : } [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:05.007.158 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_520] Added global python symbol: {const_utils : } [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.007.665 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 4, Shape: NoShape), AbstractTuple{ element[0]: AbstractScalar(Type: Int64, Value: 0, Shape: NoShape), element[1]: AbstractScalar(Type: Int64, Value: 1, Shape: NoShape), sequence_nodes: {@✓3↓flatten_521:CNode_522{[0]: ValueNode MakeTuple, [1]: ValueNode 0, [2]: ValueNode 1}, elements_use_flags: {ptr: 0x2838bcc0, value: [const vector]{0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: _number_in_tuple_520 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.011.068 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Flatten. node: @↓✓3↓flatten_523:CNode_524{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput}, new_node: @↓✓3↓flatten_523:CNode_525{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.011.141 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Flatten. node: @↓✓3↓flatten_523:CNode_524{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput}, new node: @↓✓3↓flatten_523:CNode_524{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.011.469 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_419:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_419:CNode_526{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.011.522 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_419:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_419:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:05.012.774 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_not_equal_scalar_527] Added global python symbol: {F : } [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.013.134 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 2, Shape: NoShape), AbstractScalar(Type: Int64, Value: 2, Shape: NoShape)}, g: _not_equal_scalar_527 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.013.879 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_528:CNode_529{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_528:CNode_530{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.013.950 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_528:CNode_529{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_528:CNode_529{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.015.919 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_532:CNode_533{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_532:CNode_534{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.015.990 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_532:CNode_533{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_532:CNode_533{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.017.180 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_535:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_535:CNode_536{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias, [3]: ValueNode 0} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.017.250 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_535:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_535:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias, [3]: ValueNode 0} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.017.516 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_537{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.017.581 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.017.779 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_411:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_411:CNode_538{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.017.828 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_411:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_411:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.018.705 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_539:CNode_540{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_539:CNode_541{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.018.773 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_539:CNode_540{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_539:CNode_540{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.020.568 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_542:CNode_543{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_542:CNode_544{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.020.637 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_542:CNode_543{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_542:CNode_543{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.021.846 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_545:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_545:CNode_546{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias, [3]: ValueNode 0} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.021.918 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_545:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_545:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias, [3]: ValueNode 0} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.022.203 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_547{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.022.253 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.022.431 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_403:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_403:CNode_548{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.022.480 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_403:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_403:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.023.332 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_549:CNode_550{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_549:CNode_551{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.023.402 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_549:CNode_550{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_549:CNode_550{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.025.197 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_552:CNode_553{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_552:CNode_554{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.025.267 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_552:CNode_553{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_552:CNode_553{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.026.512 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_555:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_555:CNode_556{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias, [3]: ValueNode 0} [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.026.584 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_555:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_555:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias, [3]: ValueNode 0} [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.027.487 [mindspore/ccsrc/pipeline/jit/ps/action.cc:282] AbstractAnalyze] function call depth: 0, simulate call depth: 0 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.027.620 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:222] Run] Specialize set top func graph context: {FuncGraph: mindspore_train_dataset_helper__DataWrapper_construct_460 Args: [0]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [1]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [2]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [3]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [4]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [5]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [6]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [7]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), Parent: } [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.035.264 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end abstract_specialize action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.035.322 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pack_expand action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.035.436 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pack_expand action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.035.466 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start auto_monad action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.036.500 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end auto_monad action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.036.550 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start inline action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.036.570 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end inline action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.036.590 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pre_auto_parallel action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.036.616 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pre_auto_parallel action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.036.647 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pipeline_split action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.036.664 [mindspore/ccsrc/pipeline/jit/ps/pipeline_split.cc:247] PipelineSplit] Only auto_parallel and semi_auto_parallel support pipeline split. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.036.675 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pipeline_split action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.036.693 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start optimize action. [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:05.039.795 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [fill_557] Added global python symbol: {cast_ : Prim[Cast]} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:05.040.030 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] fill_557 update var `value` with node @fill_557:value{[0]: CNode_558, [1]: param_value, [2]: param_type} [INFO] PARSER(164043,ffffa1f54440,python):2024-01-10-11:41:05.040.258 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [fill_557] Added global python symbol: {fillv2_ : Prim[FillV2]} [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.046.316 [mindspore/ccsrc/frontend/parallel/pynative_shard/pynative_shard.cc:334] PynativeShard] Only auto_parallel and semi_auto_parallel support pynative shard [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.046.378 [mindspore/ccsrc/frontend/parallel/step_parallel.cc:3009] StepParallel] Strategies would be ignored in data_parallel, shard() only valid in [semi_]auto_parallel. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.048.938 [mindspore/ccsrc/pipeline/jit/ps/action.cc:282] AbstractAnalyze] function call depth: 0, simulate call depth: 0 [INFO] ANALYZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.049.796 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:222] Run] Specialize set top func graph context: {FuncGraph: 382_mindspore_train_dataset_helper__DataWrapper_construct_559 Args: [0]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [1]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [2]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [3]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [4]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [5]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [6]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), [7]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x8de62b0, value: ValueAny), value: ValueAny), Parent: } [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.052.398 [mindspore/ccsrc/frontend/parallel/pynative_shard/pynative_shard.cc:334] PynativeShard] Only auto_parallel and semi_auto_parallel support pynative shard [INFO] OPTIMIZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.054.627 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] OPTIMIZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.055.068 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] OPTIMIZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.055.412 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.055.488 [mindspore/ccsrc/frontend/parallel/cache_embedding/cache_embedding.cc:702] AddCacheEmbedding] Parameters are all not cache enable. [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.055.934 [mindspore/ccsrc/frontend/parallel/pass/assign_add_opt.cc:120] AssignAddOpt] parallel::g_device_manager is not initialized. [INFO] OPTIMIZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.055.995 [mindspore/ccsrc/frontend/optimizer/comm_op_reuse_tag.cc:59] AddCommOpReuseTag] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.056.019 [mindspore/ccsrc/frontend/parallel/pass/overlap_opt_shard_in_pipeline.cc:70] OverlapOptShardInPipeline] parallel::g_device_manager is not initialized. [INFO] OPTIMIZER(164043,ffffa1f54440,python):2024-01-10-11:41:05.056.040 [mindspore/ccsrc/frontend/optimizer/grouped_pairwise_exchange_alltoall.cc:673] SetGroupedPairwiseExchangeAllToAll] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.056.061 [mindspore/ccsrc/frontend/parallel/pass/overlap_gradmatmul_and_gradallreduce.cc:358] OverlapGradMatmulAndGradAllreduce] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.056.078 [mindspore/ccsrc/frontend/parallel/pass/split_matmul_comm_elementwise_fp.cc:184] SplitMatmulCommElementwiseFp] SplitMatmulCommElementwiseFp is only support under [semi_]auto_parallel, skip it. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.056.109 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end optimize action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.056.132 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start auto_monad_reorder action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.056.254 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end auto_monad_reorder action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.056.281 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start get_jit_bprop_graph action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.056.295 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end get_jit_bprop_graph action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.056.312 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start eliminate_special_op_node action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.056.894 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end eliminate_special_op_node action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.056.941 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start validate action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.057.055 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end validate action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.057.093 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start distribtued_split action. [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.057.116 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:372] GenerateStrategy] Current parallel mode is data_parallel [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.057.128 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:384] GenerateStrategy] Generated distributed strategy is 1 [INFO] PARALLEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.057.260 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:1270] Run] All nodes are on this precoess so there's no need to build and split distributed graph. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.057.277 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end distribtued_split action. [INFO] PROFILER(164043,ffffa1f54440,python):2024-01-10-11:41:05.057.310 [mindspore/ccsrc/plugin/device/ascend/hal/profiler/parallel_strategy_profiling.cc:48] IsProfilingParallelStrategyEnabled] Profiling parallel strategy is disabled. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.057.329 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start task_emit action. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.057.474 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.057.495 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.057.507 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1252] SetRunMode] Run graph mode with kernel by kernel by configuration. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.057.550 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:546] CompileGraphs] Status record: start compile function graph: 458_382_mindspore_train_dataset_helper__DataWrapper_construct_560 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.057.679 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unify_mindir_pm_0_erase_invalid_micro_depend in 1.95 us [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.161 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.186 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.314 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.330 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.351 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.364 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.381 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.393 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.408 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.420 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.445 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.462 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.478 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.493 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.507 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.520 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.534 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.546 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.560 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.571 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.587 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.598 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.612 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.625 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.675 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.689 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.705 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.717 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.732 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.744 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.760 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.771 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.787 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.808 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.825 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.836 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.852 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.865 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.884 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.895 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.910 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.921 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.937 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.948 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.963 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.974 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.058.990 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.000 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.016 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.026 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.041 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.052 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.067 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.078 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.093 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.104 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.124 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.136 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.155 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.166 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.182 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.193 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.209 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.224 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.241 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.252 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.299 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:727] CompileGraph] Compile graph: 458_382_mindspore_train_dataset_helper__DataWrapper_construct_560, Split segments size: 2 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.337 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:762] CompileGraphFromSegment] Compile normal segment, the first node: @458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:CNode_561{[0]: ValueNode Load, [1]: param_fc3.bias, [2]: ValueNode U} [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.521 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:421] CompileGraph] Status record: start compile graph. [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.059.555 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:1940] ConstructKernelGraph] Create graph: 3 [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.060.177 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:2697] ConstructOutput] Output:@458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:562{[0]: ValueNode Depend, [1]: 562, [2]: CNode_563} [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.061.161 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:193] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:3 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.061.667 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 35.63 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.061.975 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 232.65 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.062.402 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:154] CommonUnifyMindIR] start common unify mindir opt graph:3 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.062.811 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.063.029 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 214.68 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.063.059 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.063.098 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 34.91 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.063.120 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.063.135 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 14.63 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.063.149 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.063.345 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 188.96 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.063.793 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:79] BackendCommonOptimization] Status record: start common optimization. graph id: 3 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.064.207 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: add_dynamic_shape_attr [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.064.503 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_0_add_dynamic_shape_attr in 290.83 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.064.536 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.064.589 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_1_convert_dynamic_broadcast_to in 48.53 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.064.805 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_2_convert_const_input_to_attr in 190.04 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.064.983 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_3_custom_op_const_input_to_attr in 147.35 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.065.135 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 125.31 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.065.601 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 432.6 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.065.640 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 8.09 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.065.814 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_7_flatten_concat_fission in 143.22 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.065.959 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 115.7 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.066.095 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_9_broadcast_to_fusion in 108.12 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.066.287 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 148.98 us [WARNING] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.066.715 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:92] BackendCommonOptimization] [PROF]backend_common_optimization costs 2923 usec. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.066.749 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:99] BackendCommonOptimization] Status record: end common optimization. graph id: 3 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.067.368 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_0_renorm_split in 138.2 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.067.404 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.067.748 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_1_reduce_axis_update in 335.22 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.067.776 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim HistogramFixedWidth not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.067.799 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_2_histogram_fixed_width_fusion in 21.43 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.067.815 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim ClipByNorm not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.067.829 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_3_clip_by_norm_fission in 12.97 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.067.843 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.067.856 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_4_tensor_arry_add_flow_cond1 in 12.35 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.067.870 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayGather not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.067.883 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_5_tensor_arry_add_flow_cond2 in 11.94 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.067.896 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.067.909 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_6_ge_tensor_arry_cast_index in 11.72 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.067.922 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArray not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.067.936 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_7_tensor_array_prepare in 11.59 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.067.948 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.067.982 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_8_space_to_batch_nd_attr_update in 30.08 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.067.997 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.068.037 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_9_batch_to_space_nd_attr_update in 36.32 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.068.055 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: avg_pool_grad_unify_mindir [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.068.094 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_10_avg_pool_grad_unify_mindir in 35.46 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.068.246 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_11_adam_weight_decay_unify_mindir in 127.45 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.068.388 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_12_cdist_fission in 113.66 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.068.527 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_13_cdist_grad_fission in 110.34 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.068.692 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_14_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 136.31 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.068.868 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_15_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 146.29 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.069.565 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_16_sparse_softmax_cross_entropy_with_logits_unify_mindir in 652.57 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.069.778 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_17_dropout_unify_mindir1 in 162.5 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.069.812 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.069.976 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_18_dropoutgrad_unify_mindir in 157.07 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.006 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.028 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_19_neighbor_exchange_unify_mindir in 20.52 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.043 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.058 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_20_neighbor_exchange_v2_unify_mindir in 13.27 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.074 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.088 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_21_neighbor_exchange_v2_grad_unify_mindir in 12.99 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.106 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.136 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_22_all_to_all_unify_mindir in 26.92 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.156 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim QuantDTypeCast not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.171 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_23_quant_dtype_cst_adjust in 14.87 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.187 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim FSEDecode not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.208 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_24_fse_decode_adjust in 19.64 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.223 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.240 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_25_bn_split in 15.97 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.253 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.316 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_26_bn_grad_unify_mindir in 58.48 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.339 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.358 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_27_bn_grad_split in 18.94 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.372 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.386 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_28_batchnorm_to_bninfer in 12.3 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.399 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.413 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_29_batchnormgrad_to_bninfergrad in 11.85 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.427 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.444 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_30_batch_norm_grad_infer_fission in 15.41 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.739 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_31_ascend_mindir_op_adapter in 268.4 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.771 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_32_DropoutGenMask in 1.36 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.070.948 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_33_lamb_fission_ge in 153.65 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.071.304 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_34_print_insert_placeholder_for_tensor_name in 324.82 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.071.521 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_35_getnext_for_ge in 171.67 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.071.693 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_36_sync_bn_split in 137.91 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.071.865 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_37_sync_bn_grad_split in 140.43 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.072.027 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_38_adaptive_max_pool2d_ge_fusion in 132.66 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.072.193 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_39_avg_pool_grad_for_ge in 136.7 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.072.220 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:260] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.072.620 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_40_FlashAttentionFusionV1 in 394.9 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.072.652 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:374] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.073.045 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_41_FlashAttentionFusionV1 in 388.93 us [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.074.445 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164043,ffffa1f54440,python):2024-01-10-11:41:05.074.486 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:05.074.505 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:132] GetRunMode] RunMode::kKernelMode [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.075.006 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 455.08 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.075.721 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_0_seed_adapter in 434.92 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.075.757 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: env_op_attr_update [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.076.019 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_1_env_op_attr_update in 254.54 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.076.196 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_2_process call inline in 142.67 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.076.295 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_3_expander_fallback in 69.05 us [WARNING] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.076.530 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:139] GEBackendOptimizeACL] [PROF]ascend_backend_optimize_acl costs 1262 usec. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.076.593 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] GetNext is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.076.673 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2D is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.076.865 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPool is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.076.943 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2D is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.077.080 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPool is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.077.200 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.077.438 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.077.648 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.078.093 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] SoftmaxCrossEntropyWithLogits is not defined in opdef. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.078.556 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 60.98 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.078.982 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 377.61 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.079.370 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 345.12 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.079.951 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 537.83 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.080.478 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_4_insert_type_transform_op in 482.32 us [WARNING] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.080.735 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:179] GEBackendOptimizeACLAfterKernelSelect] [PROF]ascend_backend_optimize_acl_after_kernel_select costs 2289 usec. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.080.822 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_0_DropoutGenMask in 0.81 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.080.993 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_1_cse in 136.18 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.081.900 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_redundant_op in 865.59 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.082.084 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_3_eliminate_maketuple_getitem in 142.81 us [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.082.119 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_4_MergeTransData in 1.35 us [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.082.543 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive GetNext [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:41:05.082.725 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:467] ConvertAny] Value: ValueTuple [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.082.873 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive GetNext [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.082.906 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2D [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:41:05.082.983 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.083.080 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2D [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.083.106 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.083.182 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.083.205 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPool [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.083.298 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPool [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.083.325 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2D [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:41:05.083.380 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.083.462 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2D [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.083.485 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.083.560 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.083.585 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPool [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.083.673 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPool [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.083.699 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Flatten [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.083.778 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Flatten [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.083.804 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.083.912 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.083.939 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.084.059 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.084.089 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.084.158 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.084.184 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.084.270 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.084.297 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.084.389 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.084.417 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.084.485 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.084.509 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.084.593 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.084.619 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.084.713 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.084.741 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.084.828 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.084.854 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.084.945 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.084.990 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive OneHot [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.085.137 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive OneHot [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.085.166 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive SoftmaxCrossEntropyWithLogits [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.085.278 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive SoftmaxCrossEntropyWithLogits [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.085.305 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReduceMean [INFO] KERNEL(164043,ffffa1f54440,python):2024-01-10-11:41:05.085.422 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReduceMean [WARNING] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:05.085.447 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:262] CreateKernel] [PROF]create_kernel costs 2925 usec. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.085.592 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1_6, index: 0 to input Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2_6, index: 0 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.085.632 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/Reshape-op0_6, index: 0 to input Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6, index: 0 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.085.665 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/Reshape-op1_6, index: 0 to input Default/GetNext-op1_6, index: 1 [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.085.742 [mindspore/ccsrc/backend/common/session/session_basic.cc:1140] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 3 start [INFO] DEBUG(164043,ffffa1f54440,python):2024-01-10-11:41:05.085.764 [mindspore/ccsrc/debug/summary/summary.cc:33] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 3 start [INFO] DEBUG(164043,ffffa1f54440,python):2024-01-10-11:41:05.085.780 [mindspore/ccsrc/debug/summary/summary.cc:54] RecurseSetSummaryNodesForAllGraphs] The total summary nodes is: 0 for graph: 3 [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.008 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:891] PrintGraphExecuteOrder] Graph 3 execution order: [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.061 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[0], node name[Default/GetNext-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:outputs{[0]: ValueNode GetNext}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.114 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[1], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv1-Conv2d/Conv2D-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode Conv2D, [1]: 562, [2]: CNode_564}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.162 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[2], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op4_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReLU, [1]: 562}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.201 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[3], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op3_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode MaxPool, [1]: 562}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.242 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[4], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/Conv2D-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode Conv2D, [1]: 562, [2]: CNode_565}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.276 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[5], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op5_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReLU, [1]: 562}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.308 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[6], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode MaxPool, [1]: 562}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.347 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[7], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_Flatten, [1]: 562}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.393 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[8], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/MatMul-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode MatMul, [1]: 562, [2]: CNode_566}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.438 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[9], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/BiasAdd-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_BiasAdd, [1]: 562, [2]: CNode_567, [3]: ValueNode 0}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.472 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[10], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op6_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReLU, [1]: 562}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.510 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[11], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/MatMul-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode MatMul, [1]: 562, [2]: CNode_568}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.556 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[12], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/BiasAdd-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_BiasAdd, [1]: 562, [2]: CNode_569, [3]: ValueNode 0}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.601 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[13], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op7_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReLU, [1]: 562}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.642 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[14], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/MatMul-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode MatMul, [1]: 562, [2]: CNode_570}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.682 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[15], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_BiasAdd, [1]: 562, [2]: CNode_561, [3]: ValueNode 0}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.734 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[16], node name[Default/Reshape-op0_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_Reshape, [1]: 562, [2]: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10])}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.775 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[17], node name[Default/Reshape-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_Reshape, [1]: 562, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[32])}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.836 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[18], node name[Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/OneHot-op0_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_OneHot, [1]: 562, [2]: ValueNode Tensor(shape=[], dtype=Int64, value=10), [3]: ValueNode Tensor(shape=[], dtype=Float32, value=1), [4]: ValueNode Tensor(shape=[], dtype=Float32, value=0), [5]: ValueNode -1}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.883 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[19], node name[Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/SoftmaxCrossEntropyWithLogits-op0_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode SoftmaxCrossEntropyWithLogits, [1]: 562, [2]: 562}] [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.086.933 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[20], node name[Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/ReduceMean-op0_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReduceMean, [1]: 562, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), [3]: ValueNode false}] [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.087.074 [mindspore/ccsrc/backend/common/somas/somas.cc:143] Assign] Start Somas Assign for graph 3 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.087.206 [mindspore/ccsrc/backend/common/somas/somas.cc:151] Assign] Somas Configure success, configuration info: Device Name: Ascend Run by execution order: 1 Enable debug log: 0 Debug log path: . [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.087.224 [mindspore/ccsrc/backend/common/somas/somas.cc:154] Assign] Start Initialize SOMAS Model [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.087.698 [mindspore/ccsrc/backend/common/somas/somas.cc:1065] GraphOutputProcess] Set 3 graph output tensors' aligned size to 0. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.087.887 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 7 output 8 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.087.916 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 16 output 17 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.087.933 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 1 output 18 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.087.949 [mindspore/ccsrc/backend/common/somas/somas.cc:1135] RefNodeProcess] Special Tensor total size: RefNode: input 53760 output 52608 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.087.975 [mindspore/ccsrc/backend/common/somas/somas.cc:555] InitSomasModel] Created 1 streams (0 groups), 21 nodes, 23 tensors, 3 union tensors lists, and 0 contiguous tensors lists [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.135 [mindspore/ccsrc/backend/common/somas/somas.cc:157] Assign] End Initialize SOMAS Model [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.151 [mindspore/ccsrc/backend/common/somas/somas.cc:176] Assign] Start Computing Conflict Matrix [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.163 [mindspore/ccsrc/backend/common/somas/somas.cc:1286] ComputeBasicMatrix] Start Conflict Computing (Bitset Model) [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.179 [mindspore/ccsrc/backend/common/somas/somas.cc:1291] ComputeBasicMatrix] Start Bitset [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.198 [mindspore/ccsrc/backend/common/somas/somas.cc:1299] ComputeBasicMatrix] Start Path Computing [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.211 [mindspore/ccsrc/backend/common/somas/somas.cc:1307] ComputeBasicMatrix] End Path Computing [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.221 [mindspore/ccsrc/backend/common/somas/somas.cc:1309] ComputeBasicMatrix] Start Tensor Relation Computing [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.274 [mindspore/ccsrc/backend/common/somas/somas.cc:1462] ComputeMultiTensorConflicts] Start Computing Conflicts Pairs, tensors list size is 23 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.298 [mindspore/ccsrc/backend/common/somas/somas.cc:1469] ComputeMultiTensorConflicts] End Computing Conflicts Pairs (time taken 0ms) [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.314 [mindspore/ccsrc/backend/common/somas/somas.cc:1367] ComputeBasicMatrix] End Basic Conflict Computing (Bitset Model)(time taken 0ms) [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.339 [mindspore/ccsrc/backend/common/somas/somas.cc:178] Assign] End Computing Conflict Matrix [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.351 [mindspore/ccsrc/backend/common/somas/somas.cc:1533] Solve] Somas Assign start... [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.370 [mindspore/ccsrc/backend/common/somas/somas.cc:1555] Solve] Start Solving [INFO] PRE_ACT(164043,fffea37fe0f0,python):2024-01-10-11:41:05.088.500 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 20 tensors [INFO] PRE_ACT(164043,fffec0ff90f0,python):2024-01-10-11:41:05.088.531 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 20 tensors [INFO] PRE_ACT(164043,fffea2ffd0f0,python):2024-01-10-11:41:05.088.510 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 20 tensors [INFO] PRE_ACT(164043,fffea3fff0f0,python):2024-01-10-11:41:05.088.521 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 20 tensors [INFO] PRE_ACT(164043,fffec0ff90f0,python):2024-01-10-11:41:05.088.633 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 1/4 1205760 Bytes (0.00112295 GB) Shared Objects size(>), index(<) bestfit [INFO] PRE_ACT(164043,fffea37fe0f0,python):2024-01-10-11:41:05.088.607 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 2/4 1205760 Bytes (0.00112295 GB) Shared Objects size(>), index(<) smallest [INFO] PRE_ACT(164043,fffea2ffd0f0,python):2024-01-10-11:41:05.088.655 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 4/4 1205760 Bytes (0.00112295 GB) Single Object size(>), index(<) smallest [INFO] PRE_ACT(164043,fffea3fff0f0,python):2024-01-10-11:41:05.088.675 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 3/4 1205760 Bytes (0.00112295 GB) Single Object size(>), index(<) bestfit [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.700 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:176] Solving] SOMAS SOLVER RESUME: [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.717 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:177] Solving] Best Solution:[1/4] [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.738 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:178] Solving] Best result:1205760 Bytes 0.00112295 GB (0.00112295 GB + 0 GB from lifelong tensors) [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.750 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:181] Solving] Best timing:0 ms [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.761 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:182] Solving] Best algorithm: Shared Objects [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.775 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:183] Solving] Best sorting strategy: size(>), index(<) [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.786 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:184] Solving] Best offset strategy: bestfit [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.796 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:185] Solving] Time elapsed: 0 ms [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.808 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:186] Solving] Spread:0 %% [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.847 [mindspore/ccsrc/backend/common/somas/somas.cc:1564] Solve] End Solving [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.874 [mindspore/ccsrc/backend/common/somas/somas.cc:2096] GenGraphStatisticInfo] Lower Bound: 1205760 (0.00112295 GB), Upper Bound: 2039296 (0.00189924 GB) [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.887 [mindspore/ccsrc/backend/common/somas/somas.cc:2099] GenGraphStatisticInfo] Total Dynamic Size (Upper Bound): 2039296 Theoretical Optimal Size (Lower Bound): 1205760 Total Workspace Size: 0 Total Communication Input Tensor Size: 0 Total Communication Output Tensor Size: 0 Total LifeLong All Tensor Size: 0 Total LifeLong Start Tensor Size: 0 Total LifeLong End Tensor Size: 2560 Reused Size(Allocate Size): 0 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.899 [mindspore/ccsrc/backend/common/somas/somas.cc:1583] Solve] Somas Assign end. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.941 [mindspore/ccsrc/backend/common/somas/somas.cc:380] UpdateSomasResultToGraph] Merged Block size: 3 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.966 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 0, offset: 602624, size: 602624 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.978 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 1, offset: 0, size: 602624 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.088.989 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 2, offset: 1205248, size: 512 [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:05.089.000 [mindspore/ccsrc/backend/common/somas/somas.cc:189] Assign] Somas Allocate end. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:05.089.012 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:310] DoSomas] Somas allocate success for graph 3 somas size: 1205760 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.089.109 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:801] CreateDeviceAddress] Status record: start create device address. graph id: 3 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:05.089.362 [mindspore/ccsrc/runtime/device/device_address_utils.cc:454] CreateValueNodeDeviceAddress] No device address for value node:Default/data-9_6, debug name:ValueNode U [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:05.090.281 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/GetNext-op1_6, index is 1; cur kernel is Default/Reshape-op1_6, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:05.090.336 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/GetNext-op1_6, index is 1; cur kernel is Default/Reshape-op1_6, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:05.090.367 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6, index is 0; cur kernel is Default/Reshape-op0_6, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:05.090.392 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6, index is 0; cur kernel is Default/Reshape-op0_6, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:05.090.412 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2_6, index is 0; cur kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1_6, index is 0 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:05.090.437 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2_6, index is 0; cur kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1_6, index is 0 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.090.457 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:808] CreateDeviceAddress] Status record: end create device address. graph id: 3 [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.090.546 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1129] CacheGraphOutputToFrontNodeWithIndex] Get graph backend output nodes. [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.090.593 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1137] CacheGraphOutputToFrontNodeWithIndex] Get graph front output nodes. [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.090.646 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1155] CacheGraphOutputToFrontNodeWithIndex] Backend output: Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/ReduceMean-op0_6 with index: 0 map to front node: Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits-op0_6 with index: 0 [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.090.661 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1155] CacheGraphOutputToFrontNodeWithIndex] Backend output: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6 with index: 0 map to front node: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op0_6 with index: 0 [INFO] SESSION(164043,ffffa1f54440,python):2024-01-10-11:41:05.090.673 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1155] CacheGraphOutputToFrontNodeWithIndex] Backend output: Default/GetNext-op1_6 with index: 1 map to front node: Default/GetNext-op0_6 with index: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.090.713 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:507] CompileGraph] Status record: end compile graph. graph id: 3 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.090.997 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:805] CompileGraphFromSegment] Compile cut segment, the cut node: @458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:CNode_571{[0]: ValueNode Return, [1]: 562} [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.091.179 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:521] Transform] Graph(kernel_graph_3) transforms actor begin, strategy:pipeline_with_execution_order [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.091.245 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2619] PersistDeviceTensorForValueNode] The device address is not exist: ValueNode_572(U) [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.091.339 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1101] BuildDataSourceActor] Create queue data source actor: kernel_graph_3_DeviceDSActor_3 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.091.630 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1232] BuildLoopCountActor] Create loop count actor: kernel_graph_3_LoopCountActor [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.091.660 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1265] BuildOutputActor] Create output actor: kernel_graph_3_OutputActor [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.091.681 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1285] BuildDataPrepareActor] Create data prepare actor: kernel_graph_3_DataPrepareActor [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.091.776 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:866] CacheGraphOutputToActor] Cache graph 3 output node:Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6 debug string:@kernel_graph_3:562{[0]: ValueNode PrimFunc_BiasAdd, [1]: 562, [2]: CNode_561, [3]: ValueNode 0} with index:0 to actor:Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6, from front node:Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op0_6 debug string:@458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:562{[0]: ValueNode PrimFunc_BiasAdd, [1]: 562, [2]: CNode_561, [3]: ValueNode 0} with index:0 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.091.800 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:809] AddSomasInfoForGraphOutput] The graph 3 output node:Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6 with index: 0 somas enable or not: 1, somas offset: 545280, aligned size: 1536 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.091.894 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:866] CacheGraphOutputToActor] Cache graph 3 output node:Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/ReduceMean-op0_6 debug string:@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReduceMean, [1]: 562, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), [3]: ValueNode false} with index:0 to actor:Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/ReduceMean-op0_6, from front node:Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits-op0_6 debug string:@458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:562{[0]: ValueNode SparseSoftmaxCrossEntropyWithLogits, [1]: 562, [2]: 562} with index:0 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.091.912 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:809] AddSomasInfoForGraphOutput] The graph 3 output node:Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/ReduceMean-op0_6 with index: 0 somas enable or not: 1, somas offset: 546816, aligned size: 512 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.091.952 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:866] CacheGraphOutputToActor] Cache graph 3 output node:Default/GetNext-op1_6 debug string:@kernel_graph_3:outputs{[0]: ValueNode GetNext} with index:1 to actor:kernel_graph_3_DeviceDSActor_3, from front node:Default/GetNext-op0_6 debug string:@458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:outputs{[0]: ValueNode GetNext} with index:1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.091.976 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1518] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_3 start. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.092.023 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1520] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_3 end. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.092.705 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 1_actor_set_kernel_graph_3_memory_actor_insert in 18.35 us [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.092.744 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 2_actor_set_kernel_graph_3_invalid_data_arrow_elimination in 2.97 us [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.092.834 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 3_actor_set_kernel_graph_3_multi_actor_fusion in 68.22 us [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.092.864 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 4_actor_set_kernel_graph_3_batch_data_arrow_fusion in 7.37 us [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.092.884 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:554] Transform] Graph(kernel_graph_3) transforms actor end. [WARNING] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.093.326 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:612] CompileGraphs] [PROF]compile_func_graph costs 35748 usec. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.093.374 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:615] CompileGraphs] Status record: end compile function graph: 458_382_mindspore_train_dataset_helper__DataWrapper_construct_560, produce actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.093.408 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end task_emit action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.093.434 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:268] SetLoopCount] Change vm_loop_flag to 0, set loop_size to 1 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.093.453 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start execute action. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.093.475 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end execute action. TotalTime = 0.221507, [19] [parse]: 0.00192875 [symbol_resolve]: 0.093707, [1] [Cycle 1]: 0.0931031, [1] [resolve]: 0.0930797 [graph_reusing]: 7.316e-05 [meta_unpack_prepare]: 0.00089019 [pre_cconv]: 3.346e-05 [abstract_specialize]: 0.066618 [pack_expand]: 0.00013528 [auto_monad]: 0.00106686 [inline]: 3.567e-05 [pre_auto_parallel]: 4.979e-05 [pipeline_split]: 3.975e-05 [optimize]: 0.0194316, [35] [py_interpret_to_execute]: 0.00023548 [rewriter_before_opt_a]: 0.00132066 [opt_a]: 0.0147701, [2] [Cycle 1]: 0.0105605, [30] [expand_dump_flag]: 1.82e-05 [switch_simplify]: 0.00034793 [a_1]: 0.00341095 [recompute_prepare]: 3.684e-05 [updatestate_depend_eliminate]: 0.00033928 [updatestate_assign_eliminate]: 5.01e-05 [updatestate_loads_eliminate]: 0.00022956 [parameter_eliminate]: 3.44e-06 [a_2]: 0.00068856 [accelerated_algorithm]: 3.18e-05 [pynative_shard]: 4.712e-05 [auto_parallel]: 4.26e-06 [parallel]: 3.301e-05 [merge_comm]: 1.832e-05 [allreduce_fusion]: 1.035e-05 [virtual_dataset]: 1.985e-05 [get_grad_eliminate_]: 1.672e-05 [virtual_output]: 1.609e-05 [merge_forward]: 2.874e-05 [cell_reuse_recompute_pass]: 5.10001e-07 [cell_reuse_handle_not_recompute_node_pass]: 4.938e-05 [meta_fg_expand]: 2.818e-05 [after_resolve]: 2.541e-05 [a_after_grad]: 2.222e-05 [renormalize]: 0.00442237 [real_op_eliminate]: 2.615e-05 [auto_monad_grad]: 4.54e-06 [auto_monad_eliminator]: 0.00011006 [cse]: 0.00015807 [a_3]: 0.00013976 [Cycle 2]: 0.00144442, [30] [expand_dump_flag]: 1.25e-06 [switch_simplify]: 1.757e-05 [a_1]: 0.00026211 [recompute_prepare]: 1.561e-05 [updatestate_depend_eliminate]: 2.975e-05 [updatestate_assign_eliminate]: 2.754e-05 [updatestate_loads_eliminate]: 2.566e-05 [parameter_eliminate]: 1.92e-06 [a_2]: 0.00031207 [accelerated_algorithm]: 3.039e-05 [pynative_shard]: 4.581e-05 [auto_parallel]: 4.61e-06 [parallel]: 6.06e-06 [merge_comm]: 1.255e-05 [allreduce_fusion]: 8.13e-06 [virtual_dataset]: 1.888e-05 [get_grad_eliminate_]: 1.645e-05 [virtual_output]: 1.64e-05 [merge_forward]: 2.337e-05 [cell_reuse_recompute_pass]: 3.6e-07 [cell_reuse_handle_not_recompute_node_pass]: 4.833e-05 [meta_fg_expand]: 1.817e-05 [after_resolve]: 2.491e-05 [a_after_grad]: 2.184e-05 [renormalize]: 6.00048e-08 [real_op_eliminate]: 1.645e-05 [auto_monad_grad]: 2.26e-06 [auto_monad_eliminator]: 6.071e-05 [cse]: 9.67e-05 [a_3]: 0.00013626 [py_interpret_to_execute_after_opt_a]: 3.457e-05 [slice_cell_reuse_recomputed_activation]: 1.88e-06 [rewriter_after_opt_a]: 0.00085778 [convert_after_rewriter]: 3.446e-05 [order_py_execute_after_rewriter]: 2.057e-05 [opt_b]: 0.00072955, [1] [Cycle 1]: 0.00072419, [7] [b_1]: 0.00045744 [b_2]: 2.018e-05 [updatestate_depend_eliminate]: 2.356e-05 [updatestate_assign_eliminate]: 2.234e-05 [updatestate_loads_eliminate]: 2.475e-05 [renormalize]: 4.084e-05 [cse]: 9.271e-05 [cconv]: 3.298e-05 [opt_after_cconv]: 0.00029646, [1] [Cycle 1]: 0.00029171, [7] [c_1]: 6.357e-05 [parameter_eliminate]: 1.49e-06 [updatestate_depend_eliminate]: 2.588e-05 [updatestate_assign_eliminate]: 2.569e-05 [updatestate_loads_eliminate]: 2.498e-05 [cse]: 8.006e-05 [renormalize]: 3.507e-05 [remove_dup_value]: 9.541e-05 [tuple_transform]: 0.00023884, [1] [Cycle 1]: 0.00023435, [3] [d_1]: 0.00013226 [d_2]: 5.064e-05 [renormalize]: 3.399e-05 [add_cache_embedding]: 6.2e-05 [add_recomputation]: 0.00027 [cse_after_recomputation]: 9.311e-05, [1] [Cycle 1]: 8.768e-05, [1] [cse]: 8.155e-05 [environ_conv]: 2.513e-05 [label_micro_interleaved_index]: 1.81999e-06 [label_fine_grained_interleaved_index]: 1.3e-06 [assign_add_opt]: 3.345e-05 [slice_recompute_activation]: 1.52e-06 [micro_interleaved_order_control]: 1.11001e-06 [full_micro_interleaved_order_control]: 1.04e-06 [comp_comm_scheduling]: 1.22e-06 [reorder_send_recv_between_fp_bp]: 1.02e-06 [comm_op_add_attrs]: 6.69999e-07 [add_comm_op_reuse_tag]: 2.284e-05 [overlap_opt_shard_in_pipeline]: 1.71e-05 [grouped_pairwise_exchange_alltoall]: 1.338e-05 [overlap_recompute_and_grad_model_parallel]: 1.1e-06 [overlap_grad_matmul_and_grad_allreduce]: 1.223e-05 [split_matmul_comm_elemetwise]: 1.664e-05 [split_layernorm_comm]: 1.09e-06 [process_send_recv_for_ge]: 6.69999e-07 [handle_group_info]: 5.30003e-07 [auto_monad_reorder]: 0.00014025 [get_jit_bprop_graph]: 2.595e-05 [eliminate_special_op_node]: 0.00061351 [validate]: 0.00014299 [distribtued_split]: 0.00020118 [task_emit]: 0.0360951 [execute]: 3.27e-05 Sums parse : 0.001929s : 0.89% symbol_resolve.resolve : 0.093080s : 42.85% graph_reusing : 0.000073s : 0.03% meta_unpack_prepare : 0.000890s : 0.41% pre_cconv : 0.000033s : 0.02% abstract_specialize : 0.066618s : 30.67% pack_expand : 0.000135s : 0.06% auto_monad : 0.001067s : 0.49% inline : 0.000036s : 0.02% pre_auto_parallel : 0.000050s : 0.02% pipeline_split : 0.000040s : 0.02% optimize.py_interpret_to_execute : 0.000235s : 0.11% optimize.rewriter_before_opt_a : 0.001321s : 0.61% optimize.opt_a.expand_dump_flag : 0.000019s : 0.01% optimize.opt_a.switch_simplify : 0.000366s : 0.17% optimize.opt_a.a_1 : 0.003673s : 1.69% optimize.opt_a.recompute_prepare : 0.000052s : 0.02% optimize.opt_a.updatestate_depend_eliminate : 0.000369s : 0.17% optimize.opt_a.updatestate_assign_eliminate : 0.000078s : 0.04% optimize.opt_a.updatestate_loads_eliminate : 0.000255s : 0.12% optimize.opt_a.parameter_eliminate : 0.000005s : 0.00% optimize.opt_a.a_2 : 0.001001s : 0.46% optimize.opt_a.accelerated_algorithm : 0.000062s : 0.03% optimize.opt_a.pynative_shard : 0.000093s : 0.04% optimize.opt_a.auto_parallel : 0.000009s : 0.00% optimize.opt_a.parallel : 0.000039s : 0.02% optimize.opt_a.merge_comm : 0.000031s : 0.01% optimize.opt_a.allreduce_fusion : 0.000018s : 0.01% optimize.opt_a.virtual_dataset : 0.000039s : 0.02% optimize.opt_a.get_grad_eliminate_ : 0.000033s : 0.02% optimize.opt_a.virtual_output : 0.000032s : 0.01% optimize.opt_a.merge_forward : 0.000052s : 0.02% optimize.opt_a.cell_reuse_recompute_pass : 0.000001s : 0.00% optimize.opt_a.cell_reuse_handle_not_recompute_node_pass : 0.000098s : 0.04% optimize.opt_a.meta_fg_expand : 0.000046s : 0.02% optimize.opt_a.after_resolve : 0.000050s : 0.02% optimize.opt_a.a_after_grad : 0.000044s : 0.02% optimize.opt_a.renormalize : 0.004422s : 2.04% optimize.opt_a.real_op_eliminate : 0.000043s : 0.02% optimize.opt_a.auto_monad_grad : 0.000007s : 0.00% optimize.opt_a.auto_monad_eliminator : 0.000171s : 0.08% optimize.opt_a.cse : 0.000255s : 0.12% optimize.opt_a.a_3 : 0.000276s : 0.13% optimize.py_interpret_to_execute_after_opt_a : 0.000035s : 0.02% optimize.slice_cell_reuse_recomputed_activation : 0.000002s : 0.00% optimize.rewriter_after_opt_a : 0.000858s : 0.39% optimize.convert_after_rewriter : 0.000034s : 0.02% optimize.order_py_execute_after_rewriter : 0.000021s : 0.01% optimize.opt_b.b_1 : 0.000457s : 0.21% optimize.opt_b.b_2 : 0.000020s : 0.01% optimize.opt_b.updatestate_depend_eliminate : 0.000024s : 0.01% optimize.opt_b.updatestate_assign_eliminate : 0.000022s : 0.01% optimize.opt_b.updatestate_loads_eliminate : 0.000025s : 0.01% optimize.opt_b.renormalize : 0.000041s : 0.02% optimize.opt_b.cse : 0.000093s : 0.04% optimize.cconv : 0.000033s : 0.02% optimize.opt_after_cconv.c_1 : 0.000064s : 0.03% optimize.opt_after_cconv.parameter_eliminate : 0.000001s : 0.00% optimize.opt_after_cconv.updatestate_depend_eliminate : 0.000026s : 0.01% optimize.opt_after_cconv.updatestate_assign_eliminate : 0.000026s : 0.01% optimize.opt_after_cconv.updatestate_loads_eliminate : 0.000025s : 0.01% optimize.opt_after_cconv.cse : 0.000080s : 0.04% optimize.opt_after_cconv.renormalize : 0.000035s : 0.02% optimize.remove_dup_value : 0.000095s : 0.04% optimize.tuple_transform.d_1 : 0.000132s : 0.06% optimize.tuple_transform.d_2 : 0.000051s : 0.02% optimize.tuple_transform.renormalize : 0.000034s : 0.02% optimize.add_cache_embedding : 0.000062s : 0.03% optimize.add_recomputation : 0.000270s : 0.12% optimize.cse_after_recomputation.cse : 0.000082s : 0.04% optimize.environ_conv : 0.000025s : 0.01% optimize.label_micro_interleaved_index : 0.000002s : 0.00% optimize.label_fine_grained_interleaved_index : 0.000001s : 0.00% optimize.assign_add_opt : 0.000033s : 0.02% optimize.slice_recompute_activation : 0.000002s : 0.00% optimize.micro_interleaved_order_control : 0.000001s : 0.00% optimize.full_micro_interleaved_order_control : 0.000001s : 0.00% optimize.comp_comm_scheduling : 0.000001s : 0.00% optimize.reorder_send_recv_between_fp_bp : 0.000001s : 0.00% optimize.comm_op_add_attrs : 0.000001s : 0.00% optimize.add_comm_op_reuse_tag : 0.000023s : 0.01% optimize.overlap_opt_shard_in_pipeline : 0.000017s : 0.01% optimize.grouped_pairwise_exchange_alltoall : 0.000013s : 0.01% optimize.overlap_recompute_and_grad_model_parallel : 0.000001s : 0.00% optimize.overlap_grad_matmul_and_grad_allreduce : 0.000012s : 0.01% optimize.split_matmul_comm_elemetwise : 0.000017s : 0.01% optimize.split_layernorm_comm : 0.000001s : 0.00% optimize.process_send_recv_for_ge : 0.000001s : 0.00% optimize.handle_group_info : 0.000001s : 0.00% auto_monad_reorder : 0.000140s : 0.06% get_jit_bprop_graph : 0.000026s : 0.01% eliminate_special_op_node : 0.000614s : 0.28% validate : 0.000143s : 0.07% distribtued_split : 0.000201s : 0.09% task_emit : 0.036095s : 16.62% execute : 0.000033s : 0.02% Time group info: ------[substitution.] 0.090900 1195 0.01% : 0.000007s : 5: substitution.depend_value_elim 0.01% : 0.000007s : 8: substitution.float_tuple_getitem_switch 97.82% : 0.088915s : 180: substitution.getattr_setattr_resolve 0.02% : 0.000019s : 40: substitution.graph_param_transform 1.58% : 0.001434s : 75: substitution.inline 0.01% : 0.000006s : 14: substitution.less_batch_normalization 0.01% : 0.000009s : 48: substitution.load_eliminater 0.26% : 0.000239s : 428: substitution.meta_unpack_prepare 0.01% : 0.000005s : 4: substitution.minmaximum_grad 0.00% : 0.000003s : 40: substitution.partial_unused_args_eliminate 0.01% : 0.000007s : 64: substitution.remove_not_recompute_node 0.01% : 0.000005s : 16: substitution.replace_old_param 0.01% : 0.000014s : 15: substitution.switch_simplify 0.03% : 0.000023s : 8: substitution.tuple_list_convert_item_index_to_positive 0.01% : 0.000009s : 8: substitution.tuple_list_get_item_const_eliminator 0.02% : 0.000014s : 8: substitution.tuple_list_get_item_depend_reorder 0.04% : 0.000040s : 15: substitution.tuple_list_get_item_eliminator 0.01% : 0.000014s : 8: substitution.tuple_list_get_set_item_eliminator 0.06% : 0.000052s : 104: substitution.updatestate_pure_node_eliminater 0.09% : 0.000078s : 107: substitution.updatestate_useless_node_eliminater ------[renormalize.] 0.004415 2 51.57% : 0.002277s : 1: renormalize.infer 48.43% : 0.002138s : 1: renormalize.specialize ------[replace.] 0.002742 256 0.51% : 0.000014s : 2: replace.depend_value_elim 78.02% : 0.002139s : 163: replace.getattr_setattr_resolve 16.37% : 0.000449s : 75: replace.inline 4.84% : 0.000133s : 15: replace.switch_simplify 0.26% : 0.000007s : 1: replace.tuple_list_get_item_eliminator ------[match.] 0.090125 256 0.00% : 0.000001s : 2: match.depend_value_elim 98.39% : 0.088672s : 163: match.getattr_setattr_resolve 1.59% : 0.001434s : 75: match.inline 0.02% : 0.000014s : 15: match.switch_simplify 0.00% : 0.000004s : 1: match.tuple_list_get_item_eliminator ------[func_graph_cloner_run.] 0.005434 106 68.87% : 0.003743s : 29: func_graph_cloner_run.FuncGraphClonerGraph 31.13% : 0.001692s : 77: func_graph_cloner_run.FuncGraphSpecializer ------[meta_graph.] 0.000000 0 ------[manager.] 0.000000 0 ------[pynative] 0.000000 0 ------[others.] 0.100365 122 1.33% : 0.001337s : 69: opt.transform.opt_a 0.44% : 0.000444s : 23: opt.transform.opt_b 92.73% : 0.093069s : 2: opt.transform.opt_resolve 0.82% : 0.000819s : 1: opt.transforms.meta_unpack_prepare 4.36% : 0.004375s : 20: opt.transforms.opt_a 0.06% : 0.000061s : 1: opt.transforms.opt_after_cconv 0.02% : 0.000019s : 1: opt.transforms.opt_b 0.18% : 0.000180s : 2: opt.transforms.opt_trans_graph 0.06% : 0.000061s : 3: opt.transforms.special_op_eliminate [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.094.112 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1385] Run] End [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.094.144 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:846] SaveCompiledGraph] Save compiled func graph(458_382_mindspore_train_dataset_helper__DataWrapper_construct_560) phase(eval.1704858064714363904.281470244928208.0)! [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.094.175 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:864] SaveCompiledGraph] End save compiled func graph! [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.094.189 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:942] CleanCompileRes] Clean compile resource start [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.096.535 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:956] CleanCompileRes] Clean compile resource end [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.096.573 [mindspore/ccsrc/pipeline/jit/ps/event_message_print.cc:37] PrintEventMessage] End compiling '_DataWrapper.construct'. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.096.588 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1039] CompileInner] Finish compiling. [WARNING] ME(164043:281473398948928,MainProcess):2024-01-10-11:41:05.971.20 [mindspore/parallel/_utils.py:259] You are suggested to use mindspore.context.set_auto_parallel_context(parameter_broadcast=True) or mindspore.common.set_seed() to share parameters among multi-devices. [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.098.642 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.098.697 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.098.752 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.098.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), output index: 0 device address:0x2b523220 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.099.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Float32, value=1), output index: 0 device address:0x2840aad0 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.099.231 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode -1 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.099.317 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode false [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.099.396 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode 1 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.099.486 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Float32, value=0), output index: 0 device address:0x2ad3e2c0 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.099.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Int64, value=10), output index: 0 device address:0x2842ed50 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.099.661 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10]), output index: 0 device address:0x2af40820 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.099.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[1], dtype=Int64, value=[32]), output index: 0 device address:0x2b36cb60 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.099.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode 0 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:05.099.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] GE(164043,python):2024-01-10-11:41:05.176.884 [graph_var_manager.cc:1424][EVENT]167074 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164043,python):2024-01-10-11:41:05.176.977 [graph_manager.cc:1248][EVENT]167074 PreRun:PreRun start: graph node size 2, session id 32, graph id 31, graph name online. [INFO] ATRACE(164043,python):2024-01-10-11:41:05.177.428 [atrace_api.c:28](tid:167074) AtraceCreate start [INFO] ATRACE(164043,python):2024-01-10-11:41:05.177.486 [trace_rb_log.c:84](tid:167074) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164043,python):2024-01-10-11:41:05.177.500 [atrace_api.c:32](tid:167074) AtraceCreate end [INFO] TDT(164043,python):2024-01-10-11:41:05.177.517 [client_manager.cpp:157][SetProfilingCallback][tid:167074] [TsdClient] set profiling callback success [INFO] GE(164043,python):2024-01-10-11:41:05.178.020 [parallel_partitioner.cc:165][EVENT]167074 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [16] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.178.062 [parallel_partitioner.cc:178][EVENT]167074 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.178.108 [graph_prepare.cc:1378][EVENT]167074 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.178.265 [graph_manager.cc:1050][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [172] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.178.292 [graph_manager.cc:1052][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.178.361 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.178.392 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.178.447 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [43] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.178.460 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.178.507 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.178.521 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.178.534 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.178.634 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.178.680 [graph_manager.cc:1054][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [375] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.178.891 [graph_manager.cc:1055][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [196] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.179.687 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:41:05.179.718 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.179.729 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.179.743 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [222] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.179.752 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.179.761 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:41:05.179.769 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.179.778 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.179.786 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.071 [graph_manager.cc:1056][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2161] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.181.135 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.153 [graph_prepare.cc:1982][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [45] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.181.462 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:41:05.181.485 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.495 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.505 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferShapePass is [136] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.514 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.523 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [4] [INFO] GE(164043,python):2024-01-10-11:41:05.181.531 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.539 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [5] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.548 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of InferValuePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.595 [graph_prepare.cc:1983][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [429] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.181.618 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.181.629 [graph_prepare.cc:1984][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.181.643 [graph_prepare.cc:1985][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.181.657 [graph_prepare.cc:1986][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.181.668 [graph_prepare.cc:1987][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.181.683 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.181.719 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.181.733 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.181.808 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.823 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.832 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.841 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.849 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.858 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.866 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.874 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.882 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.891 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.899 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.907 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.915 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.923 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.940 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.948 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.181.969 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.181.982 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.182.011 [graph_prepare.cc:1988][EVENT]167074 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [333] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.182.025 [graph_manager.cc:1065][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [918] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.193.937 [graph_manager.cc:1077][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11893] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.194.006 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.194.063 [graph_manager.cc:1080][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [86] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.196.663 [graph_manager.cc:1081][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [2583] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.196.705 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.196.719 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.196.731 [graph_manager.cc:1082][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [33] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.196.758 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.196.772 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.196.786 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.196.817 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.196.832 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.196.845 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.196.857 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.196.893 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [26] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.196.922 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.196.948 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [17] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.196.973 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.196.987 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.196.999 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.010 [graph_manager.cc:2700][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [257] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.093 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.197.106 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.197.115 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.197.123 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.197.132 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.197.141 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CastRemovePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.197.149 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.197.157 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.197.166 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.197.174 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.197.182 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.197.191 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.197.199 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [6] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.197.207 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.197.216 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.197.225 [graph_manager.cc:2741][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [197] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.234 [graph_manager.cc:2752][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.263 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.276 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [0] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.295 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [5] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.308 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.323 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.339 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.357 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.372 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.385 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.395 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.413 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.424 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.442 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.454 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.463 [graph_manager.cc:2810][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [203] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.488 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.197.501 [graph_manager.cc:2821][EVENT]167074 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [29] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.526 [graph_manager.cc:1087][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [780] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.660 [graph_manager.cc:1088][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [118] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.738 [graph_manager.cc:1089][EVENT]167074 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [58] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.758 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.772 [graph_manager.cc:1097][EVENT]167074 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164043,python):2024-01-10-11:41:05.197.801 [graph_manager.cc:3325][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.913 [engine_place.cc:144][EVENT]167074 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.927 [engine_place.cc:144][EVENT]167074 Run:The time cost of aicpu_ascend_kernel::CheckSupported is [41] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.197.991 [graph_manager.cc:3351][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [176] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.198.006 [graph_manager.cc:3364][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.198.062 [engine_partitioner.cc:1139][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [14] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.198.077 [engine_partitioner.cc:1142][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.198.192 [engine_partitioner.cc:1148][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [106] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.198.218 [engine_partitioner.cc:1155][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.198.256 [engine_partitioner.cc:1164][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [27] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.198.288 [graph_manager.cc:3405][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [271] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.198.305 [graph_manager.cc:3412][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.199.983 [graph_manager.cc:3422][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [1665] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.019 [graph_manager.cc:3428][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.133 [graph_manager.cc:3467][EVENT]167074 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [94] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.151 [graph_manager.cc:3377][EVENT]167074 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [2134] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.167 [graph_manager.cc:1106][EVENT]167074 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [2372] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.180 [graph_manager.cc:1115][EVENT]167074 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:41:05.200.203 [graph_manager.cc:1130][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.233 [graph_manager.cc:1131][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.265 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.281 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.290 [graph_manager.cc:2837][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [31] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.347 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.200.360 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.200.369 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.200.378 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.200.387 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [4] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.200.395 [base_pass.cc:339][EVENT]167074 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [2] [INFO] GE(164043,python):2024-01-10-11:41:05.200.405 [graph_manager.cc:2864][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [99] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.416 [graph_manager.cc:2872][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.435 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.448 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.463 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.476 [compile_nodes_pass.cc:88][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.486 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.497 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.564 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [57] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.587 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [11] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.602 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.619 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.638 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.652 [graph_manager.cc:2927][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [220] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.664 [graph_manager.cc:2937][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.695 [graph_manager.cc:2943][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.710 [graph_manager.cc:2950][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.864 [graph_manager.cc:2958][EVENT]167074 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.200.898 [graph_manager.cc:1132][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [641] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.201.004 [graph_manager.cc:1135][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [92] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.201.040 [graph_manager.cc:2975][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.201.146 [graph_manager.cc:2981][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [92] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.201.162 [pass_manager.cc:82][EVENT]167074 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.201.173 [graph_manager.cc:2986][EVENT]167074 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.201.182 [graph_manager.cc:1136][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [161] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.201.275 [graph_manager.cc:3555][EVENT]167074 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [68] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.201.328 [engine_partitioner.cc:1139][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [12] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.201.342 [engine_partitioner.cc:1142][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [1] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.201.425 [engine_partitioner.cc:1148][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [73] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.201.447 [engine_partitioner.cc:1155][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [8] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.201.478 [engine_partitioner.cc:1164][EVENT]167074 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [20] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.201.498 [graph_builder.cc:865][EVENT]167074 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [195] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.201.568 [graph_builder.cc:288][EVENT]167074 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [54] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.201.770 [graph_builder.cc:293][EVENT]167074 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [177] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.201.945 [model_builder.cc:1133][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [84] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.202.187 [block_mem_assigner.cc:4069][EVENT]172190 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164043,python):2024-01-10-11:41:05.202.192 [block_mem_assigner.cc:4069][EVENT]172191 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164043,python):2024-01-10-11:41:05.202.576 [graph_mem_assigner.cc:2166][EVENT]167074 SetInputOffset:[IMAS]AfterAssignMemory : online_31 memoffset[132096], memtype[2], theory_min[264192], zero_copy[132096], total_size[132096], no_reuse[132096], streams[1], topo_mode[DFS], mop[], io_reuse[0:0], alloc_mode[] [INFO] GE(164043,python):2024-01-10-11:41:05.202.663 [model_builder.cc:1144][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [696] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.202.687 [model_builder.cc:1152][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.202.703 [model_builder.cc:1157][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.202.808 [model_builder.cc:1167][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [93] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.202.827 [model_builder.cc:1174][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [6] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.202.847 [model_builder.cc:1180][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [7] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.202.883 [model_builder.cc:1184][EVENT]167074 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.202.903 [graph_builder.cc:304][EVENT]167074 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [1109] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:41:05.203.002 [logger.cc:1071] 167074 ModelBindStream: model_id=64, stream_id=1345, flag=0. [INFO] GE(164043,python):2024-01-10-11:41:05.203.066 [task_generator.cc:804][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [4] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.203.126 [task_generator.cc:805][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [42] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.203.627 [task_generator.cc:814][EVENT]167074 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [485] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.203.641 [task_generator.cc:954][EVENT]167074 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [579] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.203.695 [task_generator.cc:967][EVENT]167074 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [31] micro second. [INFO] RUNTIME(164043,python):2024-01-10-11:41:05.203.713 [logger.cc:1084] 167074 ModelUnbindStream: model_id=64, stream_id=1345, [INFO] GE(164043,python):2024-01-10-11:41:05.203.767 [graph_builder.cc:310][EVENT]167074 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [850] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.203.874 [graph_manager.cc:1152][EVENT]167074 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2673] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.203.890 [graph_manager.cc:1164][EVENT]167074 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164043,python):2024-01-10-11:41:05.203.921 [graph_manager.cc:1271][EVENT]167074 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [25982] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.203.936 [graph_manager.cc:1272][EVENT]167074 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164043,python):2024-01-10-11:41:05.204.242 [atrace_api.c:93](tid:167074) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:41:05.204.262 [atrace_api.c:95](tid:167074) AtraceDestroy end [INFO] GE(164043,python):2024-01-10-11:41:05.204.893 [model_introduction.cc:236][EVENT]167074 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(164043,python):2024-01-10-11:41:05.204.917 [model_introduction.cc:294][EVENT]167074 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(164043,python):2024-01-10-11:41:05.204.930 [model_introduction.cc:366][EVENT]167074 Data:model io_info size:116 [INFO] GE(164043,python):2024-01-10-11:41:05.208.379 [graph_converter.cc:838][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1284] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.208.563 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [135] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.208.983 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [393] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.209.072 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [61] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.209.091 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [83] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.209.132 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [30] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.209.165 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.209.197 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.209.270 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CEM is [60] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.209.332 [copy_flow_launch_fuse.cc:395][EVENT]167074 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [48] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.209.346 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [63] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.209.380 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [21] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.209.407 [base_optimizer.cc:70][EVENT]167074 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.209.425 [graph_converter.cc:849][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1004] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.209.624 [graph_converter.cc:853][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [189] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.210.279 [graph_converter.cc:857][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [637] micro second. [INFO] GE(164043,python):2024-01-10-11:41:05.210.408 [graph_converter.cc:862][EVENT]167074 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [90] micro second. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.215.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 1 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.215.989 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 1, execution time: 117.141 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.216.136 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.216.218 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.216.253 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.216.281 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.217.532 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.217.592 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.217.631 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.217.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:05.220.521 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 2 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.220.618 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 2, execution time: 2.919 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.220.696 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.220.745 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.220.773 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.220.795 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.221.597 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.221.648 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.221.681 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.221.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.223.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 3 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.223.945 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 3, execution time: 2.14539 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.224.016 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.224.060 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.224.088 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.224.111 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.224.883 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.224.932 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.224.960 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.225.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:05.227.204 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 4 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.227.304 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 4, execution time: 2.29259 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.227.380 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.227.427 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.227.456 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.227.478 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.228.268 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.228.319 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.228.348 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.228.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.230.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 5 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.230.539 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 5, execution time: 2.1387 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.230.612 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.230.660 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.230.689 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.230.711 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.231.478 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.231.528 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.231.556 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.231.695 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.233.623 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 6 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.233.740 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 6, execution time: 2.13307 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.233.820 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.233.881 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.233.909 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.233.931 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.234.699 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.234.751 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.234.779 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.234.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:05.236.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 7 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.236.986 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 7, execution time: 2.15612 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.237.059 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.237.106 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.237.137 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.237.164 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.237.957 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.238.011 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.238.043 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.238.186 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.240.109 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 8 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.240.199 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 8, execution time: 2.10268 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.240.274 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.240.320 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.240.352 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.240.378 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.241.143 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.241.195 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.241.227 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.241.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.243.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 9 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.243.418 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 9, execution time: 2.13482 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.243.493 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.243.537 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.243.566 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.243.588 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.244.351 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.244.401 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.244.431 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.244.572 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.246.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 10 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.246.606 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 10, execution time: 2.12038 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.246.678 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.246.721 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.246.750 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.246.772 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.247.530 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.247.581 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.247.614 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.247.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.249.739 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 11 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.249.829 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 11, execution time: 2.16312 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.249.902 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.249.946 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.249.977 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.249.999 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.250.756 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.250.806 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.250.835 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.250.978 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.252.916 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 12 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.252.994 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 12, execution time: 2.10519 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.253.067 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.253.111 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.253.138 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.253.163 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.253.970 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.254.022 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.254.055 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.254.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:05.256.111 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 13 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.256.199 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 13, execution time: 2.09324 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.256.273 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.256.318 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.256.350 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.256.373 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.257.141 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.257.191 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.257.223 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.257.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.259.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 14 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.259.402 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 14, execution time: 2.12252 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.259.475 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.259.522 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.259.552 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.259.577 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.260.344 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.260.395 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.260.424 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.260.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.262.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 15 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.262.645 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 15, execution time: 2.16805 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.262.718 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.262.762 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.262.791 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.262.813 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.263.582 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.263.632 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.263.661 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.263.798 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:05.265.809 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 16 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.265.897 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 16, execution time: 2.18523 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.265.970 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.266.014 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.266.043 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.266.065 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.266.827 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.266.878 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.266.908 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.267.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.268.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 17 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.269.080 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 17, execution time: 2.12198 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.269.154 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.269.198 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.269.226 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.269.248 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.270.042 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.270.098 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.270.128 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.270.270 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.272.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 18 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.272.299 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 18, execution time: 2.11796 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.272.374 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.272.418 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.272.448 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.272.473 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.273.244 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.273.294 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.273.323 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:05.273.459 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.275.440 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 19 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.275.543 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 19, execution time: 2.16797 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.275.618 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.275.663 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.275.693 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.275.731 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.276.495 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.276.545 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.276.577 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.276.715 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:05.278.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 20 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.279.026 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 20, execution time: 2.39651 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.279.105 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.279.153 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.279.182 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.279.209 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.279.985 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.280.035 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.280.065 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.280.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.282.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 21 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.282.343 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 21, execution time: 2.22376 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.282.421 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.282.470 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.282.501 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.282.523 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.283.292 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.283.343 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.283.379 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.283.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:05.285.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 22 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.285.578 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 22, execution time: 2.14364 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.285.653 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.285.749 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.285.783 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.285.810 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.286.590 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.286.640 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.286.673 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.286.817 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.288.766 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 23 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.288.865 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 23, execution time: 2.1397 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.288.940 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.288.986 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.289.015 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.289.039 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.289.811 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.289.863 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.289.895 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.290.039 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.291.992 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 24 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.292.076 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 24, execution time: 2.12693 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.292.153 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.292.196 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.292.225 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.292.248 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.293.026 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.293.078 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.293.108 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.293.246 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.295.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 25 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.295.300 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 25, execution time: 2.14094 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.295.378 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.295.421 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.295.450 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.295.472 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.296.238 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.296.290 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.296.318 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.296.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.298.403 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 26 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.298.496 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 26, execution time: 2.12542 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.298.572 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.298.621 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.298.653 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.298.678 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.299.481 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.299.533 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.299.568 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.299.705 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.301.622 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 27 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.301.713 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 27, execution time: 2.09015 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.301.802 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.301.848 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.301.879 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.301.904 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.302.669 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.302.720 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.302.752 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:05.302.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.304.853 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 28 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.304.936 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 28, execution time: 2.13327 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.305.015 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.305.061 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.305.094 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.305.117 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.305.908 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.305.962 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.305.993 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.306.132 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.308.054 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 29 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.308.142 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 29, execution time: 2.0944 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.308.226 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.308.269 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.308.300 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.308.322 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.309.085 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.309.137 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.309.170 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.309.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.311.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 30 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.311.352 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 30, execution time: 2.13053 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.311.427 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.311.474 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.311.505 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.311.527 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.312.293 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.312.345 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.312.374 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.312.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.314.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 31 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.314.586 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 31, execution time: 2.15801 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.314.674 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.314.722 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.314.753 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.314.778 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.315.547 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.315.598 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.315.627 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.315.769 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:05.318.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 32 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.318.327 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 32, execution time: 2.64349 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.318.406 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.318.453 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.318.481 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.318.503 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.319.276 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.319.327 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.319.355 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.319.503 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.321.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 33 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.321.670 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 33, execution time: 2.26295 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.321.780 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.321.826 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.321.854 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.321.877 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.322.634 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.322.685 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.322.715 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.322.853 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.324.811 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 34 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.324.898 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 34, execution time: 2.13243 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.324.973 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.325.016 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.325.045 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.325.067 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.326.005 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.326.057 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.326.088 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.326.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:05.328.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 35 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.328.259 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 35, execution time: 2.12009 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.328.341 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.328.386 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.328.414 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.328.440 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.329.210 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.329.260 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.329.289 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.329.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.331.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 36 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.331.494 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 36, execution time: 2.15399 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.331.569 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.331.615 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.331.647 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.331.673 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.332.438 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.332.489 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.332.518 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.332.654 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:05.334.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 37 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.334.736 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 37, execution time: 2.16759 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.334.824 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.334.871 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.334.902 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.334.927 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.335.702 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.335.752 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.335.781 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.335.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.337.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 38 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.337.930 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 38, execution time: 2.09937 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.338.004 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.338.049 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.338.080 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.338.103 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.338.889 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.338.940 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.338.973 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.339.111 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:05.341.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 39 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.341.069 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 39, execution time: 2.04255 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.341.145 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.341.210 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.341.238 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.341.260 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.342.065 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.342.117 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.342.150 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.342.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.344.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 40 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.344.634 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 40, execution time: 2.43396 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.344.710 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.344.754 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.344.781 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.344.803 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.345.560 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.345.612 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.345.641 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.345.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.348.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 41 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.348.416 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 41, execution time: 2.72056 ms in multi thread or not: 0. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.348.495 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.348.556 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.348.586 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.348.609 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.349.382 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.349.433 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.349.462 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.349.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.351.703 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 42 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.351.795 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 42, execution time: 2.27814 ms in multi thread or not: 0. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.351.873 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.351.923 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.351.953 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.351.975 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.352.750 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.352.802 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.352.831 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.352.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.355.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 43 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.355.174 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 43, execution time: 2.28825 ms in multi thread or not: 0. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.355.251 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.355.315 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.355.346 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.355.375 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.356.148 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.356.199 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.356.231 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.356.350 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.358.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 44 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.358.605 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 44, execution time: 2.31922 ms in multi thread or not: 0. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.358.684 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.358.735 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.358.769 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.358.793 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.359.567 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.359.618 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.359.646 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.359.765 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.361.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 45 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.361.989 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 45, execution time: 2.28882 ms in multi thread or not: 0. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.362.067 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.362.130 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.362.161 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.362.188 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.362.959 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.363.011 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.363.041 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.363.158 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.365.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 46 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.365.351 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 46, execution time: 2.2561 ms in multi thread or not: 0. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.365.431 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.365.480 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.365.511 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.365.536 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.366.336 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.366.386 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.366.415 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.366.531 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.368.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 47 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.368.700 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 47, execution time: 2.23306 ms in multi thread or not: 0. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.368.779 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.368.829 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.368.877 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.368.901 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.369.677 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.369.751 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.369.780 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.369.898 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.371.968 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 48 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.372.058 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 48, execution time: 2.22416 ms in multi thread or not: 0. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.372.138 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.372.191 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.372.225 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.372.249 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.373.039 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.373.090 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.373.119 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.373.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.375.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 49 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.375.486 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 49, execution time: 2.31425 ms in multi thread or not: 0. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.375.566 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.375.618 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.375.667 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.375.692 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.376.463 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.376.515 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.376.543 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.376.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.378.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 50 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.378.865 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 50, execution time: 2.26517 ms in multi thread or not: 0. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.378.893 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:767] SetActorExecutionStrategy] Multi thread execution time cost: 2.22144 ms, single thread execution time cost: 2.31877 ms, decide to use multi thread execution or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.378.959 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.379.005 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.379.034 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.379.057 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.379.832 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.379.882 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.379.912 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.380.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:05.382.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 51 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.382.160 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 51, execution time: 2.19225 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.382.244 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.382.289 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.382.317 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.382.340 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.383.114 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.383.164 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.383.195 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.383.336 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.385.259 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 52 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.385.339 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 52, execution time: 2.09176 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.385.412 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.385.460 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.385.488 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.385.510 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.386.323 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.386.376 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.386.408 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.386.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:05.388.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 53 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.388.626 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 53, execution time: 2.16446 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.388.703 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.388.756 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.388.785 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.388.806 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.389.593 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.389.643 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.389.672 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.389.829 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.391.769 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 54 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.391.858 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 54, execution time: 2.12032 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.391.932 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.391.975 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.392.003 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.392.026 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.392.793 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.392.843 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.392.872 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.393.007 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.394.971 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 55 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.395.060 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 55, execution time: 2.13729 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.395.135 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.395.192 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.395.223 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.395.247 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.396.010 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.396.062 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.396.095 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.396.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.398.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 56 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.398.347 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 56, execution time: 2.19892 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.398.422 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.398.469 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.398.501 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.398.526 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.399.289 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.399.339 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.399.367 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.399.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.401.788 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 57 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.401.876 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 57, execution time: 2.45752 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.401.952 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.402.007 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.402.039 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.402.064 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.402.842 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.402.892 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.402.922 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:05.403.073 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.405.192 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 58 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.405.279 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 58, execution time: 2.30498 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.405.352 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.405.396 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.405.425 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.405.447 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.406.231 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.406.284 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.406.312 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.406.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.408.372 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 59 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.408.459 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 59, execution time: 2.09614 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.408.534 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.408.587 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.408.615 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.408.638 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.409.406 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.409.457 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.409.485 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.409.622 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:41:05.411.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 60 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.411.750 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 60, execution time: 2.21202 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.411.829 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.411.873 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.411.900 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.411.922 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.412.688 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.412.738 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.412.766 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.412.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:41:05.414.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 61 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.414.984 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 61, execution time: 2.16704 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.415.063 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.415.119 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.415.147 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.415.169 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.415.935 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.415.985 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.416.017 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.416.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:05.418.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 62 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.418.274 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 62, execution time: 2.2047 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.418.348 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.418.392 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.418.421 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.418.446 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.419.209 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.419.258 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.419.288 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.419.427 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.421.574 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 63 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.421.665 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 63, execution time: 2.32741 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.421.777 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.421.834 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.421.880 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.421.903 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.422.675 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.422.726 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.422.754 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.422.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.424.829 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 64 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.424.918 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 64, execution time: 2.112 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.424.995 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.425.044 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.425.074 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.425.099 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.425.915 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.425.967 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.425.998 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.426.139 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3574a0f0,python):2024-01-10-11:41:05.428.129 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 65 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.428.219 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 65, execution time: 2.16773 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.428.297 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.428.340 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.428.378 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.428.401 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.429.169 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.429.220 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.429.248 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.429.389 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.431.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 66 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.431.448 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 66, execution time: 2.14764 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.431.522 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.431.567 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.431.596 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.431.618 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.432.391 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.432.442 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.432.471 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.432.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3a74c0f0,python):2024-01-10-11:41:05.434.582 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 67 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.434.677 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 67, execution time: 2.15337 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.434.760 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.434.804 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.434.848 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.434.871 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.435.637 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.435.688 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.435.717 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.435.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.437.805 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 68 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.437.894 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 68, execution time: 2.12665 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.437.967 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.438.014 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.438.046 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.438.072 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.438.837 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.438.888 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.438.916 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.439.052 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.440.989 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 69 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.441.076 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 69, execution time: 2.10873 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.441.151 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.441.196 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.441.228 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.441.265 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.442.072 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.442.126 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.442.157 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.442.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.444.262 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 70 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.444.347 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 70, execution time: 2.1367 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.444.420 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.444.466 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.444.494 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.444.516 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.445.275 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.445.325 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.445.354 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.445.489 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:05.447.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 71 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.447.553 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 71, execution time: 2.14739 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.447.626 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.447.669 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.447.698 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.447.732 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.448.495 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.448.545 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.448.574 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.448.713 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.450.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 72 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.450.757 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 72, execution time: 2.13283 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.450.831 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.450.874 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.450.901 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.450.924 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.451.684 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.451.734 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.451.764 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.451.900 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.454.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 73 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.454.379 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 73, execution time: 2.56312 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.454.455 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.454.500 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.454.529 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.454.565 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.455.347 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.455.397 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.455.427 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.455.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.458.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 74 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.458.090 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 74, execution time: 2.61019 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.458.170 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.458.217 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.458.248 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.458.271 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.459.051 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.459.101 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.459.133 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.459.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:05.461.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 75 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.461.344 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 75, execution time: 2.1583 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.461.416 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.461.459 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.461.488 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.461.522 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.462.310 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.462.361 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.462.390 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.462.530 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe3af4d0f0,python):2024-01-10-11:41:05.464.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 76 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.464.679 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 76, execution time: 2.24001 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.464.754 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.464.798 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.464.826 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.464.848 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.465.615 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.465.664 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.465.752 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.465.897 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.467.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 77 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.467.987 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 77, execution time: 2.18095 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.468.061 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.468.105 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.468.135 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.468.157 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164043,ffffa1f54440,python):2024-01-10-11:41:05.468.936 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.468.986 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.469.015 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164043,fffe3b74e0f0,python):2024-01-10-11:41:05.469.151 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164043,fffe39f4b0f0,python):2024-01-10-11:41:05.471.176 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 78 [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:05.471.266 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 78, execution time: 2.19983 ms in multi thread or not: 1. [INFO] VM(164043,ffffa1f54440,python):2024-01-10-11:41:05.471.337 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.471.379 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.471.409 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.471.431 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty Accuracy: 0.07892628205128205 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.708.513 [mindspore/ccsrc/pipeline/jit/ps/init.cc:515] operator()] Start register... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.708.598 [mindspore/ccsrc/pipeline/jit/ps/init.cc:519] operator()] Start mindspore.profiler... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.708.671 [mindspore/ccsrc/pipeline/jit/ps/init.cc:527] operator()] Start EmbeddingCacheScheduler... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.708.697 [mindspore/ccsrc/pipeline/jit/ps/init.cc:534] operator()] Start releasing dataset handles... [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:05.708.752 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164043,ffffa1f54440,python):2024-01-10-11:41:05.708.853 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.716.124 [mindspore/ccsrc/pipeline/jit/ps/init.cc:537] operator()] End release dataset handles. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:05.716.174 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2393] FinalizeCluster] Start finalize the cluster instance. [INFO] DISTRIBUTED(164043,ffff13fff0f0,python):2024-01-10-11:41:06.631.658 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:301] Heartbeat] The heartbeat thread is finished. [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:41:06.632.008 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:131] Finalize] The compute graph node has been unregistered successfully. [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:41:06.632.118 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:533] Finalize] Delete send event loop [INFO] DISTRIBUTED(164043,ffff21c180f0,python):2024-01-10-11:41:06.632.239 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:41:06.632.397 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:209] Finalize] Stop loop succ [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:41:06.632.415 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:540] Finalize] Delete recv event loop [INFO] DISTRIBUTED(164043,ffff224190f0,python):2024-01-10-11:41:06.632.479 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:41:06.632.612 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:209] Finalize] Stop loop succ [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:41:06.632.626 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:554] Finalize] Delete connection pool. [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:41:06.632.683 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:533] Finalize] Delete send event loop [INFO] DISTRIBUTED(164043,ffff20c160f0,python):2024-01-10-11:41:06.632.779 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:41:06.632.937 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:209] Finalize] Stop loop succ [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:41:06.632.954 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:540] Finalize] Delete recv event loop [INFO] DISTRIBUTED(164043,ffff214170f0,python):2024-01-10-11:41:06.633.001 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:41:06.633.126 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:209] Finalize] Stop loop succ [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:41:06.633.140 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:554] Finalize] Delete connection pool. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:06.633.159 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2396] FinalizeCluster] End finalize the cluster instance. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:06.633.172 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2335] ClearResAtexit] Pipeline clear all resource [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:06.633.250 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:290] RecordExitStatus] Status record: system exit. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:06.637.476 [mindspore/ccsrc/runtime/graph_scheduler/rpc_node_scheduler.cc:220] Clear] Start finalizing tcp server and client for rpc actors. [INFO] RUNTIME_FRAMEWORK(164043,ffffa1f54440,python):2024-01-10-11:41:06.637.512 [mindspore/ccsrc/runtime/graph_scheduler/rpc_node_scheduler.cc:230] Clear] End finalizing tcp server and client for rpc actors. [INFO] ME(164043,ffffa1f54440,python):2024-01-10-11:41:06.637.941 [mindspore/core/mindrt/src/actor/actormgr.cc:153] Finalize] mindrt Actors finish exiting. [INFO] ME(164043,ffffa1f54440,python):2024-01-10-11:41:06.637.964 [mindspore/core/mindrt/src/actor/actormgr.cc:156] Finalize] mindrt Threads finish exiting. [INFO] ME(164043,ffffa1f54440,python):2024-01-10-11:41:06.655.127 [mindspore/core/mindrt/src/actor/actormgr.cc:167] Finalize] mindrt IOMGRS finish exiting. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:06.655.928 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2207] ClearResPart1] Start Finalize StreamSynchronizer... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:06.655.966 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2209] ClearResPart1] End Finalize StreamSynchronizer... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:06.657.289 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:829] ClearRes] Clean executor resource! [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:06.657.321 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2223] ClearResPart2] Start clear PyNativeExecutor... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:06.657.674 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2225] ClearResPart2] End clear PyNativeExecutor. [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:41:06.657.776 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:179] ClearGraph] Remove all graphs in GraphManager [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:06.660.405 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2241] ClearResPart2] Start clear kernel runtime... [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:06.660.441 [mindspore/ccsrc/runtime/device/kernel_runtime_manager.cc:25] ClearRuntimeResource] Release device Ascend_2 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:06.660.457 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:240] ReleaseDeviceRes] Ascend finalize start [INFO] HCCL(164043,python):2024-01-10-11:41:06.660.532 [op_base.cc:1312][164043]com is not global com [INFO] HCCP(164043,python):2024-01-10-11:41:06.660.720 [ra_host.c:1795]tid:164043,ra_socket_white_list_del(1795) : Input parameters: phy_id[2], local_ip[2.0.0.0], num[1] [INFO] HCCP(164043,python):2024-01-10-11:41:06.661.258 [ra_host.c:863]tid:164043,ra_socket_batch_close(863) : Input parameters: [0]th, phy_id[2], local_ip[2.0.0.0] [INFO] HCCP(164043,python):2024-01-10-11:41:06.661.521 [ra_host.c:863]tid:164043,ra_socket_batch_close(863) : Input parameters: [0]th, phy_id[2], local_ip[2.0.0.0] [INFO] HCCP(164043,python):2024-01-10-11:41:06.760.288 [ra_host.c:863]tid:164043,ra_socket_batch_close(863) : Input parameters: [0]th, phy_id[2], local_ip[2.0.0.0] [INFO] HCCP(164043,python):2024-01-10-11:41:06.760.320 [ra_host.c:863]tid:164043,ra_socket_batch_close(863) : Input parameters: [1]th, phy_id[2], local_ip[2.0.0.0] [INFO] HCCP(164043,python):2024-01-10-11:41:06.760.332 [ra_host.c:863]tid:164043,ra_socket_batch_close(863) : Input parameters: [2]th, phy_id[2], local_ip[2.0.0.0] [INFO] HCCP(164043,python):2024-01-10-11:41:06.825.205 [ra_host.c:1795]tid:164043,ra_socket_white_list_del(1795) : Input parameters: phy_id[2], local_ip[2.0.0.0], num[1] [INFO] HCCP(164043,python):2024-01-10-11:41:06.825.318 [ra_host.c:1795]tid:164043,ra_socket_white_list_del(1795) : Input parameters: phy_id[2], local_ip[2.0.0.0], num[1] [INFO] HCCP(164043,python):2024-01-10-11:41:06.830.856 [ra_host.c:941]tid:164043,ra_socket_listen_stop(941) : Input parameters: [0]th, phy_id[2], local_ip[2.0.0.0] [INFO] HCCP(164043,python):2024-01-10-11:41:06.830.998 [ra_host.c:525]tid:164043,ra_socket_deinit(525) : Input parameters: phy_id[2] family[2] local_ip[2.0.0.0] [INFO] HCCP(164043,python):2024-01-10-11:41:06.831.079 [ra_host.c:349]tid:164043,ra_deinit(349) : Input parameters: phy_id[2], nic_position:[1] [INFO] HCCP(164043,python):2024-01-10-11:41:06.831.091 [ra_hdc.c:1535]tid:164043,ra_hdc_deinit(1535) : hdc deinit start! phy_id[2] [INFO] HCCP(164043,python):2024-01-10-11:41:06.831.253 [ra_hdc.c:1570]tid:164043,ra_hdc_deinit(1570) : hdc deinit OK! phy_id[2] [INFO] ATRACE(164043,python):2024-01-10-11:41:06.831.349 [atrace_api.c:93](tid:164043) AtraceDestroy start [INFO] ATRACE(164043,python):2024-01-10-11:41:06.831.374 [atrace_api.c:95](tid:164043) AtraceDestroy end [INFO] HCCL(164043,python):2024-01-10-11:41:06.853.870 [op_base.cc:1332][164043]op_base comm destroy complete,take time [193354]us, rankNum[0], rank[4294967295], deviceLogicId[2] [INFO] HCCL_ADPT(164043,ffffa1f54440,python):2024-01-10-11:41:06.853.939 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:260] FinalizeHccl] Start destroy hccl adapter for GRAPH_MODE [INFO] HCCL_ADPT(164043,ffffa1f54440,python):2024-01-10-11:41:06.853.966 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:548] FinalizeHcclExec] Start finalize hccl exec. [INFO] HCCL(164043,python):2024-01-10-11:41:06.858.871 [hcom_executor.cc:43][164043][Finalize][HcomExecutor]Hcom Excutor Finalize end. ret[0] [INFO] HCCL_ADPT(164043,ffffa1f54440,python):2024-01-10-11:41:06.858.920 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:556] FinalizeHcclExec] HcclExec destroy success [INFO] HCCL_ADPT(164043,ffffa1f54440,python):2024-01-10-11:41:06.858.955 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:388] FinalizeKernelInfoStore] Start destroy hccl kernel info store. [INFO] HCCL_ADPT(164043,ffffa1f54440,python):2024-01-10-11:41:06.858.997 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:410] FinalizeKernelInfoStore] Destroy hccl kernel info store success. [INFO] HCCL_ADPT(164043,ffffa1f54440,python):2024-01-10-11:41:06.859.010 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:437] FinalizeHcclComm] Start finalize hccl comm. [INFO] HCCL_ADPT(164043,ffffa1f54440,python):2024-01-10-11:41:06.859.052 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:273] FinalizeHccl] Destroy hccl adapter success. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:06.859.065 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:710] DestroyHccl] Hccl destroy successful. [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:06.859.116 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:783] operator()] Common mem pool info: Total allocated mem:1024M, peak used mem:4M, in used mem:0M, total idle mem:1023M. Block unit size:1024M, block counts:1, block[0] block size:1024M idle size:1023M [INFO] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:06.859.139 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:783] operator()] Persistent mem pool info: Total allocated mem:1024M, peak used mem:0M, in used mem:0M, total idle mem:1023M. Block unit size:1024M, block counts:1, block[0] block size:1024M idle size:1023M [WARNING] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:06.859.153 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:793] DumpDynamicMemPoolStateInfo] The dynamic memory pool total allocated mem:2048M, peak used mem:5M, in used mem:0M, total idle mem:71812935M, total eager free mem:0M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [WARNING] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:08.377.829 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:142] DeInitialize] Ascend Memory Adapter deinitialize success, statistics: Device HBM memory size: 32768M MindSpore Used memory size: 30678M MindSpore memory base address: 0x124100000000 Total Static Memory size: 2048M Total Dynamic memory size: 0M Dynamic memory size of this graph: 0M [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:08.399.624 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:274] ReleaseDeviceRes] Ascend finalize end [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:08.399.679 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2243] ClearResPart2] End clear kernel runtime. [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:41:08.399.700 [mindspore/ccsrc/distributed/collective/collective_manager.cc:379] Finalize] Begin finalize collective manager. [INFO] DISTRIBUTED(164043,fffec17fa0f0,python):2024-01-10-11:41:08.399.830 [mindspore/ccsrc/distributed/collective/collective_manager.cc:358] operator()] Start finalizing host communication lib. [INFO] DISTRIBUTED(164043,fffec17fa0f0,python):2024-01-10-11:41:08.399.861 [mindspore/ccsrc/distributed/collective/collective_manager.cc:362] operator()] End finalizing host communication lib. [INFO] DISTRIBUTED(164043,fffec17fa0f0,python):2024-01-10-11:41:08.399.875 [mindspore/ccsrc/distributed/collective/collective_manager.cc:367] operator()] Start finalizing device communication lib. [INFO] DISTRIBUTED(164043,fffec17fa0f0,python):2024-01-10-11:41:08.399.886 [mindspore/ccsrc/distributed/collective/collective_manager.cc:371] operator()] End finalizing device communication lib. [INFO] DISTRIBUTED(164043,ffffa1f54440,python):2024-01-10-11:41:08.399.908 [mindspore/ccsrc/distributed/collective/collective_manager.cc:386] Finalize] End finalize collective manager. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:08.399.936 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2258] ClearResPart2] Start clear device context... [INFO] ME(164043,ffffa1f54440,python):2024-01-10-11:41:08.399.951 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:469] ClearDeviceContexts] Release device Ascend_2 [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:41:08.399.977 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:264] DeleteGraphRunner] Delete GraphRunner success [TRACE] GE(164043,python):2024-01-10-11:41:08.399.995 [status:INIT] [ge_api.cc:463]164043 ~Session:Start to destruct session. [TRACE] GE(164043,python):2024-01-10-11:41:08.400.023 [status:RUNNING] [ge_api.cc:475]164043 ~Session:Session id is 0 [TRACE] GE(164043,python):2024-01-10-11:41:08.400.034 [status:RUNNING] [ge_api.cc:476]164043 ~Session:Destroying session [TRACE] GE(164043,python):2024-01-10-11:41:08.400.936 [status:STOP] [ge_api.cc:491]164043 ~Session:Session Destructor finished [INFO] GE_ADPT(164043,ffffa1f54440,python):2024-01-10-11:41:08.400.974 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:235] DeleteGeSession] Delete Ge Session success [TRACE] GE(164043,python):2024-01-10-11:41:08.400.993 [status:INIT] [ge_api.cc:301]164043 GEFinalize:GEFinalize start [INFO] GE(164043,python):2024-01-10-11:41:08.401.061 [execution_runtime.cc:80][EVENT]164043 FinalizeExecutionRuntime:Execution runtime finalize begin. [INFO] GE(164043,python):2024-01-10-11:41:08.401.079 [execution_runtime.cc:92][EVENT]164043 FinalizeExecutionRuntime:Execution runtime finalized. [TRACE] GE(164043,python):2024-01-10-11:41:08.401.090 [status:RUNNING] [ge_api.cc:313]164043 GEFinalize:Finalizing environment [INFO] TUNE(164043,python):2024-01-10-11:41:08.699.011 [cann_kb_pyfunc_mgr.cpp:127][CANNKB][Tid:164043]"CannKbPyfuncMgr: enter PyObjectDeinit function, reference_[1]" [INFO] TUNE(164043,python):2024-01-10-11:41:08.699.065 [cann_kb_pyfunc_mgr.cpp:138][CANNKB][Tid:164043]"CannKbPyfuncMgr: PyObjectDeinit function end successfully!" [INFO] GE(164043,python):2024-01-10-11:41:08.732.068 [gelib.cc:324][EVENT]164043 SystemFinalize:Online infer finalize GELib success. [TRACE] GE(164043,python):2024-01-10-11:41:09.113.640 [status:STOP] [ge_api.cc:341]164043 GEFinalize:GEFinalize finished [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:09.113.781 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:317] CloseTsd] Start to close tsd, ref = 1 [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:09.114.392 [mindspore/ccsrc/plugin/device/ascend/hal/device/tensorprint_utils.cc:449] DestroyTensorPrintThread] Succeed stop acl data channel for host queue [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:09.114.915 [mindspore/ccsrc/plugin/device/ascend/hal/device/tensorprint_utils.cc:375] JoinAclPrintThread] join acl tdt host receive process [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:09.114.971 [mindspore/ccsrc/plugin/device/ascend/hal/device/tensorprint_utils.cc:463] DestroyTensorPrintThread] Succeed destroy acl channel [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:09.114.996 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:119] DestoryHandler] The thread of ms_histogram_summary channel is being destroyed. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:09.115.011 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:115] ~MbufDataHandler] Channel ms_histogram_summary begins the destruction process. [INFO] DEVICE(164043,fffed9d860f0,python):2024-01-10-11:41:09.202.319 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:67] ~ScopeAclTdtDataset] AcltdtDestroyDataset succeed. [INFO] DEVICE(164043,fffec37fe0f0,python):2024-01-10-11:41:09.202.350 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:67] ~ScopeAclTdtDataset] AcltdtDestroyDataset succeed. [INFO] DEVICE(164043,fffed95850f0,python):2024-01-10-11:41:09.202.387 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:67] ~ScopeAclTdtDataset] AcltdtDestroyDataset succeed. [INFO] DEVICE(164043,fffed8d840f0,python):2024-01-10-11:41:09.202.432 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:67] ~ScopeAclTdtDataset] AcltdtDestroyDataset succeed. [INFO] DEVICE(164043,fffec3fff0f0,python):2024-01-10-11:41:09.202.440 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:67] ~ScopeAclTdtDataset] AcltdtDestroyDataset succeed. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:09.202.775 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:119] DestoryHandler] The thread of ms_scalar_summary channel is being destroyed. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:09.202.799 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:115] ~MbufDataHandler] Channel ms_scalar_summary begins the destruction process. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:09.203.113 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:119] DestoryHandler] The thread of ms_image_summary channel is being destroyed. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:09.203.131 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:115] ~MbufDataHandler] Channel ms_image_summary begins the destruction process. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:09.203.397 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:119] DestoryHandler] The thread of ms_tensor_summary channel is being destroyed. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:09.203.416 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:115] ~MbufDataHandler] Channel ms_tensor_summary begins the destruction process. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:09.203.651 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:119] DestoryHandler] The thread of ms_tensor_dump channel is being destroyed. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:09.203.670 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:115] ~MbufDataHandler] Channel ms_tensor_dump begins the destruction process. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.020 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:337] CloseTsd] Call aclrtResetDevice, destroy and close tsd successful, ret[0] [INFO] ME(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.048 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:469] ClearDeviceContexts] Release device CPU_0 [INFO] ME(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.064 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:469] ClearDeviceContexts] Release device CPU_2 [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.100 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2260] ClearResPart2] End clear device context. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.113 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2262] ClearResPart2] Start clear AnalysisResultCacheMgr... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.125 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2264] ClearResPart2] End clear AnalysisResultCacheMgr. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.136 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2266] ClearResPart2] Start clear AnalysisContext... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.147 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2268] ClearResPart2] End clear AnalysisContext... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.161 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2270] ClearResPart2] Start clear AnalysisSchedule... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.306 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2272] ClearResPart2] End clear AnalysisSchedule... [INFO] DEBUG(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.340 [mindspore/ccsrc/debug/debugger/debugger.cc:305] Reset] Release Debugger resource. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.378 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2285] ClearResPart3] Start clear ClearObjectCache... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.390 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2287] ClearResPart3] End clear ClearObjectCache... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.401 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2289] ClearResPart3] Start clear Parser... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.413 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2291] ClearResPart3] End clear Parser... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.423 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2293] ClearResPart3] Start ClearTraceStack... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.434 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2295] ClearResPart3] End ClearTraceStack... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.445 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2297] ClearResPart3] Start clear InterpretNodeRecorder... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.456 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2299] ClearResPart3] End clear InterpretNodeRecorder... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.466 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2301] ClearResPart3] Start clear parallel::entire_costgraph... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.482 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2303] ClearResPart3] End clear parallel::entire_costgraph... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.492 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2305] ClearResPart3] Start clear ProtobufLibrary... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.663 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2307] ClearResPart3] End clear ProtobufLibrary... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.678 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2309] ClearResPart3] Start clear python_adapter... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.690 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2311] ClearResPart3] End clear python_adapter. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.700 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2315] ClearSingleton] Start clear singleton... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.766 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2331] ClearSingleton] End clear singleton. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.780 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2347] ClearResAtexit] Start unload dynamic lib... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.204.801 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2349] ClearResAtexit] End unload dynamic lib... [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.578.375 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:803] DelOneNetRes] Delete one net resource start [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.584.593 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:825] DelOneNetRes] Delete one net resource end. [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.584.707 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:803] DelOneNetRes] Delete one net resource start [INFO] PIPELINE(164043,ffffa1f54440,python):2024-01-10-11:41:09.587.071 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:825] DelOneNetRes] Delete one net resource end. [WARNING] PRE_ACT(164043,ffffa1f54440,python):2024-01-10-11:41:10.044.201 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:793] DumpDynamicMemPoolStateInfo] The dynamic memory pool total allocated mem:2048M, peak used mem:5M, in used mem:0M, total idle mem:71812935M, total eager free mem:0M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [INFO] DEVICE(164043,ffffa1f54440,python):2024-01-10-11:41:10.044.316 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:136] DeInitialize] DeInitialize Ascend Memory Adapter when it is not initialize corrupted size vs. prev_size [INFO] ATRACE(164046,python):2024-01-10-11:37:02.200.186 [trace_attr.c:105](tid:164046) platform is 1. [INFO] ATRACE(164046,python):2024-01-10-11:37:02.200.368 [trace_recorder.c:114](tid:164046) use root path: /home/jenkins/ascend/atrace [INFO] ATRACE(164046,python):2024-01-10-11:37:02.200.393 [trace_signal.c:133](tid:164046) register signal handler for signo 2 succeed. [INFO] ATRACE(164046,python):2024-01-10-11:37:02.200.405 [trace_signal.c:133](tid:164046) register signal handler for signo 15 succeed. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:02.599.368 [mindspore/core/utils/ms_context.cc:225] set_backend_policy] ms set context backend policy:ge [INFO] RUNTIME(164046,python):2024-01-10-11:37:02.599.560 [runtime.cc:1159] 164046 GetAicoreNumByLevel: workingDev_=0 [INFO] RUNTIME(164046,python):2024-01-10-11:37:02.599.604 [runtime.cc:4719] 164046 GetVisibleDevices: ASCEND_RT_VISIBLE_DEVICES param was not set [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:02.600.362 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:580] SetContextSocVersion] The soc version :Ascend910A [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:02.641.207 [mindspore/ccsrc/pybind_api/ir/log_adapter_py.h:34] PyExceptionInitializer] Set exception handler [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:02.651.100 [mindspore/ccsrc/pipeline/jit/ps/init.cc:179] pybind11_init__c_expression] Start GraphExecutorPy... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:02.651.725 [mindspore/ccsrc/pipeline/jit/ps/init.cc:271] pybind11_init__c_expression] Start ParallelContext... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:02.652.291 [mindspore/ccsrc/pipeline/jit/ps/init.cc:379] pybind11_init__c_expression] Start CostModelContext... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:02.652.698 [mindspore/ccsrc/pipeline/jit/ps/init.cc:481] pybind11_init__c_expression] Start OffloadContext... [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:02.654.656 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:580] SetContextSocVersion] The soc version :Ascend910A [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:04.738.680 [mindspore/core/utils/ms_context.cc:362] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:37:04.738.756 [mindspore/ccsrc/frontend/parallel/costmodel_context.cc:30] GetInstance] Create costmodel_context [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:06.879.730 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:580] SetContextSocVersion] The soc version :Ascend910A [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:06.880.476 [mindspore/core/utils/ms_context.cc:362] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:06.880.648 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:580] SetContextSocVersion] The soc version :Ascend910A [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:06.881.171 [mindspore/core/utils/ms_context.cc:362] SetDeviceTargetFromInner] ms set context device target:Ascend [INFO] PS(164046,ffff82e63440,python):2024-01-10-11:37:06.881.420 [mindspore/ccsrc/ps/ps_context.cc:256] set_ms_role] MS_ROLE of this node is MS_WORKER [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:06.881.478 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:11 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:06.881.664 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:188] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281470735544560 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:06.881.699 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:13 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:06.881.801 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:188] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281470727151856 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:06.881.843 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:15 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:06.881.941 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:188] Initialize] Set pthread name success name:RECV_EVENT_LOOP,loop_thread_:281470718759152 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:06.881.961 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:17 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:06.882.055 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:188] Initialize] Set pthread name success name:SEND_EVENT_LOOP,loop_thread_:281470710366448 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:06.882.078 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:412] Connect] Can not found link destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:06.882.291 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:18 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:06.882.323 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:456] Connect] Connection 18 source: 127.0.0.1:46022, destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:06.882.342 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:475] Connect] Connected to destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:06.882.353 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:67] Connect] Connected to the tcp server 127.0.0.1:10969 successfully. [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:06.882.374 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:412] Connect] Can not found link destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164046,ffff0234d0f0,python):2024-01-10-11:37:06.882.377 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:18 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:06.882.465 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:19 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:06.882.493 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:456] Connect] Connection 19 source: 127.0.0.1:46024, destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:06.882.511 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:475] Connect] Connected to destination: 127.0.0.1:10969 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:06.882.522 [mindspore/ccsrc/distributed/rpc/tcp/tcp_client.cc:67] Connect] Connected to the tcp server 127.0.0.1:10969 successfully. [INFO] DISTRIBUTED(164046,ffff0334f0f0,python):2024-01-10-11:37:06.882.524 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:215] DeleteEvent] Not found event fd:19 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:06.882.860 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:209] Register] The compute graph node: 3 has been registered successfully. [WARNING] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:06.882.940 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:195] BuildCluster] Topology build timed out., retry(1/210). [INFO] DISTRIBUTED(164046,ffff0134b0f0,python):2024-01-10-11:37:06.882.998 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:247] Heartbeat] The heartbeat thread is started. [WARNING] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:09.883.021 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:197] BuildCluster] Cluster is successfully initialized. [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:09.883.050 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:255] PostProcess] Start post processing for computing graph nodes. [WARNING] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:09.883.233 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:261] PostProcess] This node 3 rank id: 3 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:09.883.263 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:268] PostProcess] Client ip address in this cluster of this compute graph node is 127.0.0.1 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:09.883.363 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:279] PostProcess] Port range assigned for this node 3 is 11190 to 12213 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:09.883.386 [mindspore/ccsrc/distributed/cluster/cluster_context.cc:133] node_num] Number of role MS_WORKER is 4 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:09.883.423 [mindspore/ccsrc/distributed/collective/collective_manager.cc:157] Initialize] Start initializing collective communication for backend: Ascend... [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:09.883.481 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:37] MsCollectiveCommLib] Global group name of MindSpore collective communication library is mccl_world_group [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:09.883.494 [mindspore/ccsrc/distributed/collective/collective_manager.cc:412] InitHostCommlib] Start initializing communication library on host side... [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:09.883.514 [mindspore/ccsrc/distributed/collective/collective_manager.cc:432] InitHostCommlib] Communication library on host side is successfully initialized. Global rank id: 3, global rank size: 4 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:09.883.528 [mindspore/ccsrc/distributed/collective/collective_manager.cc:470] AssignLocalRank] Host name for rank 3 is ascend85 [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:09.883.660 [mindspore/ccsrc/distributed/collective/collective_manager.cc:505] AssignLocalRank] The local rank id assigned for this process is 3. device_id of ms_context is set. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:09.896.028 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:624] SetRtDevice] Enter SetRtDevice, current initialize device number:0 [INFO] TDT(164046,python):2024-01-10-11:37:09.896.818 [process_mode_manager.cpp:109][OpenProcess][tid:164046] [ProcessModeManager] enter into open process deviceId[3] rankSize[0] [INFO] TDT(164046,python):2024-01-10-11:37:09.899.317 [process_mode_manager.cpp:379][InitTsdClient][tid:164046] [TsdClient] deviceId[3] begin to init hdc client [INFO] TDT(164046,python):2024-01-10-11:37:09.899.491 [version_verify.cpp:34][SetVersionInfo][tid:164046] VersionVerify: send client version to server [INFO] TDT(164046,python):2024-01-10-11:37:09.899.536 [version_verify.cpp:50][SetVersionInfo][tid:164046] send feature_info:{msg_type:35, features:{check before send aicpu package,}} [INFO] TDT(164046,python):2024-01-10-11:37:09.899.548 [version_verify.cpp:50][SetVersionInfo][tid:164046] send feature_info:{msg_type:37, features:{check before send open qs message,}} [INFO] TDT(164046,python):2024-01-10-11:37:09.899.859 [version_verify.cpp:66][PeerVersionCheck][tid:164046] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(164046,python):2024-01-10-11:37:09.899.876 [version_verify.cpp:87][ParseVersionInfo][tid:164046] VersionVerify: pass client version info success [INFO] TDT(164046,python):2024-01-10-11:37:09.899.885 [hdc_client.cpp:276][CheckHdcConnection][tid:164046] Service[2] create hdc success [INFO] TDT(164046,python):2024-01-10-11:37:09.899.900 [version_verify.cpp:120][SpecialFeatureCheck][tid:164046] VersionVerify: new type[35], supported [INFO] TDT(164046,python):2024-01-10-11:37:09.899.946 [process_mode_manager.cpp:748][GetDeviceCheckCode][tid:164046] [TsdClient][deviceId=3] [sessionId=1] wait package info respond [INFO] TDT(164046,python):2024-01-10-11:37:09.900.078 [process_mode_manager.cpp:379][InitTsdClient][tid:164046] [TsdClient] deviceId[3] begin to init hdc client [INFO] TDT(164046,python):2024-01-10-11:37:09.900.214 [version_verify.cpp:34][SetVersionInfo][tid:164046] VersionVerify: send client version to server [INFO] TDT(164046,python):2024-01-10-11:37:09.900.227 [version_verify.cpp:50][SetVersionInfo][tid:164046] send feature_info:{msg_type:35, features:{check before send aicpu package,}} [INFO] TDT(164046,python):2024-01-10-11:37:09.900.238 [version_verify.cpp:50][SetVersionInfo][tid:164046] send feature_info:{msg_type:37, features:{check before send open qs message,}} [INFO] TDT(164046,python):2024-01-10-11:37:09.900.375 [version_verify.cpp:66][PeerVersionCheck][tid:164046] VersionVerify: Check client version info, server[1230], client[1230] [INFO] TDT(164046,python):2024-01-10-11:37:09.900.387 [version_verify.cpp:87][ParseVersionInfo][tid:164046] VersionVerify: pass client version info success [INFO] TDT(164046,python):2024-01-10-11:37:09.900.396 [hdc_client.cpp:276][CheckHdcConnection][tid:164046] Service[2] create hdc success [INFO] TDT(164046,python):2024-01-10-11:37:09.900.408 [process_mode_manager.cpp:426][ConstructOpenMsg][tid:164046] [TsdClient] tsd get process sign successfully, procpid[164046] signSize[48] [INFO] TDT(164046,python):2024-01-10-11:37:09.900.419 [version_verify.cpp:112][SpecialFeatureCheck][tid:164046] VersionVerify: previous type[6], supported [INFO] TDT(164046,python):2024-01-10-11:37:09.900.440 [process_mode_manager.cpp:126][OpenProcess][tid:164046] [ProcessModeManager] deviceId[3] sessionId[1] rankSize[0], wait sub process start respond [INFO] TDT(164046,python):2024-01-10-11:37:10.113.356 [stub_process_mode_nowin.cpp:63][ProcessQueueForMdc][tid:164046] [TsdClient] it is unnecessary of current mode[0] chiptype[1] to grant queue auth to aicpusd [INFO] TDT(164046,python):2024-01-10-11:37:10.113.385 [stub_process_mode_nowin.cpp:101][OpenInHost][tid:164046] enter into OpenInHost deviceid[3] [INFO] TDT(164046,python):2024-01-10-11:37:10.113.395 [stub_process_mode_nowin.cpp:105][OpenInHost][tid:164046] host cpu not support [INFO] TDT(164046,python):2024-01-10-11:37:10.113.404 [process_mode_manager.cpp:156][OpenProcess][tid:164046] [TsdClient][deviceId=3] [sessionId=1] start hccp and computer process success [INFO] RUNTIME(164046,python):2024-01-10-11:37:10.116.154 [device.cc:340] 164046 Init: isDoubledie:0, topologytype:0 [INFO] RUNTIME(164046,python):2024-01-10-11:37:10.130.514 [npu_driver.cc:5428] 164294 GetDeviceStatus: GetDeviceStatus status=1. [INFO] ATRACE(164046,python):2024-01-10-11:37:10.130.580 [atrace_api.c:28](tid:164046) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:10.130.705 [trace_rb_log.c:84](tid:164046) [RUNTIME_ATRACE_DEV3_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:10.130.720 [atrace_api.c:32](tid:164046) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:10.130.742 [client_manager.cpp:157][SetProfilingCallback][tid:164046] [TsdClient] set profiling callback success [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:10.136.034 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:646] CreateDefaultStream] Create ascend default stream, stream id: 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:10.137.219 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:652] CreateDefaultStream] Create ascend communication stream, stream id: 1 [INFO] DEBUG(164046,ffff82e63440,python):2024-01-10-11:37:10.137.287 [mindspore/ccsrc/debug/debugger/debugger.cc:101] Debugger] Debugger got device_target: Ascend [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:10.137.446 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:116] Initialize] Device HBM Size:32768M, Device free HBM Size:32735M, Reserved HBM size for Other Components(HCCL/rts/etc.):2057M, Recommend Reserved HBM size for Other Components:2045M, User define MindSpore HBM Size:0G, MindSpore Used HBM Size:30678M. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:10.279.201 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:301] MallocFromRts] Call rtMalloc to allocate device memory Success, size: 32168214528 bytes, address start: 0x124100000000 end: 0x12487d600000 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:10.279.257 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:129] Initialize] Ascend Memory Adapter initialize success, Memory Statistics: Device HBM memory size: 32768M MindSpore Used memory size: 30678M MindSpore memory base address: 0x124100000000 Total Static Memory size: 0M Total Dynamic memory size: 0M Dynamic memory size of this graph: 0M [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:10.280.516 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:463] SetDisableReuseMemoryFlag] DISABLE_REUSE_MEMORY is not set in ENV. Now set to default value 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:10.280.544 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:500] SetHcclOptions] No hccl mode. If use hccl, make sure [RANK_TABLE_FILE,RANK_ID,DEVICE_ID] all be set in ENV. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:10.280.600 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:366] GetGeOptions] JOB_ID is not set in ENV. Now set to default value 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:10.280.616 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:384] GetGeOptions] Set proto lib path failed! [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:10.280.628 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:304] SetAscendConfig] GE topo sorting mode is: [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:10.280.641 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:316] SetAscendConfig] Set GE topo mode to memory-priority. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:10.280.653 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:321] SetAscendConfig] Set staticMemoryPolicy to default mode. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:10.280.665 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:329] SetAscendConfig] The default value of jit_compile is set to 2. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:10.280.676 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:285] SetAscendHF32Config] The default value of allow_matmul_hf32 and allow_conv_hf32 are set by CANN. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:10.280.686 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:294] SetAscendHF32Config] allow_matmul_hf32: , allow_conv_hf32: [TRACE] GE(164046,python):2024-01-10-11:37:10.280.736 [status:INIT] [ge_api.cc:144]164046 GEInitializeImpl:GEInitialize start [INFO] PROFILING(164046,python):2024-01-10-11:37:10.500.587 [msprofiler_impl.cpp:156] >>> (tid:164046) ProfNotifySetDevice called, is open: 1, devId: 3 [INFO] PROFILING(164046,python):2024-01-10-11:37:10.500.730 [platform.cpp:38] >>> (tid:164046) Profiling platform version: 1.0. [INFO] PROFILING(164046,python):2024-01-10-11:37:10.500.746 [ai_drv_dev_api.cpp:384] >>> (tid:164046) Succeeded to DrvGetApiVersion version: 0x72313 [TRACE] GE(164046,python):2024-01-10-11:37:10.550.515 [status:RUNNING] [ge_api.cc:211]164046 GEInitializeImpl:Initializing environment [INFO] GE(164046,python):2024-01-10-11:37:10.550.570 [gelib.cc:98][EVENT]164046 Initialize:[GEPERFTRACE] GE Init Start [INFO] GE(164046,python):2024-01-10-11:37:10.550.874 [gelib.cc:307][EVENT]164046 SystemInitialize:Online infer init GELib success, device id :3 [INFO] DVPP(164046,python):2024-01-10-11:37:10.904.041 [dvpp_engine.cc:41][ENGINE][Initialize:41][tid:164046]dvpp engine do not support [INFO] TUNE(164046,python):2024-01-10-11:37:10.909.584 [cann_kb_pyfunc_mgr.cpp:72][CANNKB][Tid:164046]"CannKbPyfuncMgr: Enter PyObjectInit, reference_ is 0!" [INFO] TUNE(164046,python):2024-01-10-11:37:10.909.622 [handle_manager.cpp:115][CANNKB][Tid:164046]"Start to run init functions to load dynamic python lib!" [INFO] TUNE(164046,python):2024-01-10-11:37:10.909.725 [handle_manager.cpp:407][CANNKB][Tid:164046]"Init functions of loading dynamic python lib end!" [INFO] TUNE(164046,python):2024-01-10-11:37:10.909.740 [cann_kb_pyfunc_mgr.cpp:24][CANNKB][Tid:164046]"CANN_KB_Py has already been initialized." [INFO] TUNE(164046,python):2024-01-10-11:37:10.909.835 [cann_kb_pyfunc_mgr.cpp:117][CANNKB][Tid:164046]"CannKbPyfuncMgr: Run PyObjectInit successfully!" [INFO] HCCL(164046,python):2024-01-10-11:37:22.741.087 [plugin_manager.cc:42][164046]hcom running normal mode. [INFO] DVPP(164046,python):2024-01-10-11:37:22.741.791 [dvpp_engine.cc:92][ENGINE][GetOpsKernelInfoStores:92][tid:164046]dvpp ops kernel info store do not support [INFO] DVPP(164046,python):2024-01-10-11:37:22.741.942 [dvpp_engine.cc:69][ENGINE][GetGraphOptimizerObjs:69][tid:164046]dvpp graph optimizer do not support [INFO] DVPP(164046,python):2024-01-10-11:37:23.395.677 [dvpp_ops_kernel_builder.cc:48][ENGINE][Initialize:48][tid:164046]dvpp ops kernel builder do not support [INFO] GE(164046,python):2024-01-10-11:37:23.404.598 [gelib.cc:169][EVENT]164046 Initialize:[GEPERFTRACE] The time cost of GELib::Initialize is [12853940] micro second. [TRACE] GE(164046,python):2024-01-10-11:37:23.486.416 [status:STOP] [ge_api.cc:255]164046 GEInitializeImpl:GEInitialize finished [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:23.486.631 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_res_manager.cc:168] GeSetContextOptions] Set GE atomic clean policy to 1. [TRACE] GE(164046,python):2024-01-10-11:37:23.486.706 [status:INIT] [ge_api.cc:398]164046 Session:Start to construct session. [TRACE] GE(164046,python):2024-01-10-11:37:23.486.723 [status:RUNNING] [ge_api.cc:408]164046 Session:Creating session [INFO] GE(164046,python):2024-01-10-11:37:23.487.214 [graph_var_manager.cc:1445][EVENT]164046 SetMemoryMallocSize:Total memory size is 34359738368 [INFO] GE(164046,python):2024-01-10-11:37:23.487.232 [graph_var_manager.cc:1424][EVENT]164046 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] PROFILING(164046,python):2024-01-10-11:37:23.487.606 [msprofiler_impl.cpp:156] >>> (tid:164046) ProfNotifySetDevice called, is open: 1, devId: 3 [TRACE] GE(164046,python):2024-01-10-11:37:23.488.476 [status:RUNNING] [ge_api.cc:411]164046 Session:Session id is 0 [TRACE] GE(164046,python):2024-01-10-11:37:23.488.499 [status:STOP] [ge_api.cc:420]164046 Session:Session Constructor finished [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:23.488.531 [mindspore/ccsrc/transform/graph_ir/graph_runner.cc:53] NewSession] Create new GE session success! [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:23.488.556 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:210] SetGeSession] Add a new Ge Session success [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:23.488.603 [mindspore/ccsrc/transform/graph_ir/graph_runner.cc:65] GraphRunner] ME run in ONE_DEVICE strategy mode [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:23.488.658 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:246] SetGraphRunner] Add a new GraphRunner success [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:23.488.703 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:238] InitGe] Create session and graphrunner successful. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:23.488.718 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:242] InitGe] Init ge successful, ge reference = 1. [INFO] PROFILING(164046,python):2024-01-10-11:37:23.498.276 [platform.cpp:38] >>> (tid:164046) Profiling platform version: 1.0. [INFO] PROFILING(164046,python):2024-01-10-11:37:23.498.303 [ai_drv_dev_api.cpp:384] >>> (tid:164046) Succeeded to DrvGetApiVersion version: 0x72313 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:23.498.445 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:193] Initialize] Call aclInit successfully. [TRACE] GE(164046,python):2024-01-10-11:37:23.498.593 [status:INIT] [ge_api.cc:144]164046 GEInitializeImpl:GEInitialize start [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:23.499.870 [mindspore/ccsrc/distributed/collective/collective_manager.cc:455] InitDeviceCommLib] Start initializing communication library on device side... [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:23.499.939 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:291] OpenTsd] Device id = 3, rank size = 4. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:23.507.450 [mindspore/ccsrc/plugin/device/ascend/hal/device/tensorprint_utils.cc:405] CreateChannel] For Print ops, select MBUF channel. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:23.507.484 [mindspore/ccsrc/plugin/device/ascend/hal/device/tensorprint_utils.cc:420] CreateTensorPrintThread] Success to create acl channel handle, tsd reference = 1. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:23.508.044 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:103] MbufDataHandler] Channel ms_tensor_dump begins the construction process. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:23.508.589 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:103] MbufDataHandler] Channel ms_tensor_summary begins the construction process. [INFO] DEVICE(164046,fffeba70f0f0,python):2024-01-10-11:37:23.508.627 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:161] HandleData] Channel ms_tensor_dump starts executing HandleData. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:23.508.990 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:103] MbufDataHandler] Channel ms_image_summary begins the construction process. [INFO] DEVICE(164046,fffeb9f0e0f0,python):2024-01-10-11:37:23.509.038 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:161] HandleData] Channel ms_tensor_summary starts executing HandleData. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:23.509.407 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:103] MbufDataHandler] Channel ms_scalar_summary begins the construction process. [INFO] DEVICE(164046,fffeb970d0f0,python):2024-01-10-11:37:23.509.464 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:161] HandleData] Channel ms_image_summary starts executing HandleData. [INFO] DEVICE(164046,fffeb8f0c0f0,python):2024-01-10-11:37:23.509.794 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:161] HandleData] Channel ms_scalar_summary starts executing HandleData. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:23.509.809 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:103] MbufDataHandler] Channel ms_histogram_summary begins the construction process. [INFO] HCCL_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:23.510.122 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:202] InitHccl] Start init hccl adapter. [INFO] DEVICE(164046,fffea3fff0f0,python):2024-01-10-11:37:23.510.156 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:161] HandleData] Channel ms_histogram_summary starts executing HandleData. [INFO] HCCL_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:23.510.255 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:341] InitKernelInfoStore] Start init hccl kernel info store. [INFO] HCCL_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:23.510.328 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:358] InitKernelInfoStore] Get builder ops_kernel_info_hccl [INFO] HCCL(164046,python):2024-01-10-11:37:23.510.354 [plugin_manager.cc:42][164046]hcom running normal mode. [INFO] HCCL_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:23.510.406 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:380] InitKernelInfoStore] Init hccl kernel info store success. [INFO] HCCL_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:23.510.434 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:528] InitHcclExec] Start init hccl exec. [INFO] HCCL(164046,python):2024-01-10-11:37:23.512.981 [hcom_executor.cc:32][164046][Initialize][HcomExecutor]Hcom Excutor Initialize end. ret[0] [INFO] HCCL_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:23.513.029 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:534] InitHcclExec] Hcom DynamicKernel Initialize success [INFO] HCCL_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:23.513.052 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:540] InitHcclExec] InitHcclExec success [INFO] HCCL_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:23.513.064 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:220] InitHccl] Init hccl adapter success. [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:23.513.082 [mindspore/ccsrc/distributed/collective/collective_manager.cc:458] InitDeviceCommLib] Communication library on device side is successfully initialized. [WARNING] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:23.513.098 [mindspore/ccsrc/distributed/collective/collective_manager.cc:220] CreateCommunicationGroup] Start to create communication group: hccl_world_group [const vector]{0, 1, 2, 3} [WARNING] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:23.513.383 [mindspore/ccsrc/plugin/device/cpu/hal/hardware/ms_collective_comm_lib.cc:200] QueryUniqueID] Retry to lookup the unique id for group hccl_world_group from the meta server node...Retry time: 199/200 [WARNING] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:26.513.609 [mindspore/ccsrc/distributed/collective/collective_manager.cc:278] CreateCommunicationGroup] Begin initialize communication group on the device side: hccl_world_group [INFO] HCCL(164046,python):2024-01-10-11:37:26.513.864 [externalinput.cc:310][166235]environmental variable HCCL_CONNECT_TIMEOUT is set, timeOut[600] [INFO] HCCL(164046,python):2024-01-10-11:37:26.513.911 [externalinput.cc:598][166235]environmental variable HCCL_IF_IP is not set [INFO] HCCL(164046,python):2024-01-10-11:37:26.513.926 [externalinput.cc:655][166235]environmental variable HCCL_SOCKET_IFNAME is not set, default[EmptyString] [INFO] HCCL(164046,python):2024-01-10-11:37:26.513.942 [externalinput.cc:582][166235]environmental variable HCCL_IF_BASE_PORT is not set [INFO] HCCL(164046,python):2024-01-10-11:37:26.514.030 [externalinput.cc:282][166235]environmental variable HCCL_HIGH_PERF_ENABLE is not set [INFO] HCCL(164046,python):2024-01-10-11:37:26.514.061 [op_base.cc:405][166235]Entry-HcclCommInitRootInfo:ranks[4], rank[3], rootinfo: host ip[8.92.9.85] port[60000] nicDeploy[1] identifier[8.92.9.85%enp189s0f0_60000_0_1704857845691656], deviceLogicId[3] [INFO] HCCP(164046,python):2024-01-10-11:37:26.514.461 [ra_host.c:1882]tid:166235,ra_get_ifnum(1882) : Input parameters: phy_id[0], nic_position:[0] [INFO] HCCL(164046,python):2024-01-10-11:37:26.516.495 [adapter_hccp.cc:821][166235][Get][HostIf]hrtGetIfNum success. ifAddrNum[7]. [INFO] HCCP(164046,python):2024-01-10-11:37:26.516.517 [ra_host.c:1930]tid:166235,ra_get_ifaddrs(1930) : Input parameters: phy_id[0], nic_position:[0], interface num[7] [INFO] HCCL(164046,python):2024-01-10-11:37:26.517.726 [sal.cc:383][166235]nic class[normal]: find nic[8.92.9.85%enp189s0f0] success. [INFO] HCCP(164046,python):2024-01-10-11:37:26.517.796 [ra_host.c:1722]tid:166235,ra_socket_set_white_list_status(1722) : Input parameters: enable[0] [INFO] HCCP(164046,python):2024-01-10-11:37:26.517.827 [ra_host.c:293]tid:166235,ra_init(293) : Input parameters: phy_id[3], nic_position:[0] [INFO] HCCP(164046,python):2024-01-10-11:37:26.518.134 [rs_ssl.c:1104]tid:166235,rs_ssl_init(1104) : TLS SWITCH (0) [INFO] HCCP(164046,python):2024-01-10-11:37:26.518.268 [rs_epoll.c:470]tid:166236,rs_epoll_handle(470) : pthread[epoll_pthread] is alive! [INFO] HCCP(164046,python):2024-01-10-11:37:26.518.328 [rs_epoll.c:595]tid:166237,rs_connect_handle(595) : pthread[connect_pthread] is alive! [INFO] HCCP(164046,python):2024-01-10-11:37:26.518.353 [rs.c:403]tid:166235,rs_init(403) : rs init success, chip_id[3] [INFO] TDT(164046,python):2024-01-10-11:37:26.518.381 [process_mode_manager.cpp:109][OpenProcess][tid:166235] [ProcessModeManager] enter into open process deviceId[3] rankSize[2] [INFO] TDT(164046,python):2024-01-10-11:37:26.518.733 [process_mode_manager.cpp:705][GetDeviceCheckCode][tid:166235] [ProcessModeManager][deviceId=3] aicpu package already exist in device [INFO] TDT(164046,python):2024-01-10-11:37:26.518.770 [process_mode_manager.cpp:426][ConstructOpenMsg][tid:166235] [TsdClient] tsd get process sign successfully, procpid[164046] signSize[48] [INFO] TDT(164046,python):2024-01-10-11:37:26.518.823 [process_mode_manager.cpp:126][OpenProcess][tid:166235] [ProcessModeManager] deviceId[3] sessionId[1] rankSize[2], wait sub process start respond [INFO] TDT(164046,python):2024-01-10-11:37:26.683.785 [stub_process_mode_nowin.cpp:63][ProcessQueueForMdc][tid:166235] [TsdClient] it is unnecessary of current mode[0] chiptype[1] to grant queue auth to aicpusd [INFO] TDT(164046,python):2024-01-10-11:37:26.683.802 [stub_process_mode_nowin.cpp:101][OpenInHost][tid:166235] enter into OpenInHost deviceid[3] [INFO] TDT(164046,python):2024-01-10-11:37:26.683.813 [stub_process_mode_nowin.cpp:105][OpenInHost][tid:166235] host cpu not support [INFO] TDT(164046,python):2024-01-10-11:37:26.683.821 [process_mode_manager.cpp:156][OpenProcess][tid:166235] [TsdClient][deviceId=3] [sessionId=1] start hccp and computer process success [INFO] HCCP(164046,python):2024-01-10-11:37:26.683.834 [ra_host.c:293]tid:166235,ra_init(293) : Input parameters: phy_id[3], nic_position:[1] [INFO] HCCP(164046,python):2024-01-10-11:37:26.683.875 [ra_hdc.c:1465]tid:166235,ra_hdc_init(1465) : hdc init start! logic id is 3, phy id is 3 [INFO] HCCP(164046,python):2024-01-10-11:37:26.684.170 [ra_hdc.c:1500]tid:166235,ra_hdc_init(1500) : hdc init OK! phy_id[3] [INFO] HCCP(164046,python):2024-01-10-11:37:26.686.683 [ra_host.c:454]tid:166235,ra_socket_init_v1(454) : socket init:mode=0 phy_id=3 family=2 ip=8.92.9.85 [INFO] HCCL(164046,python):2024-01-10-11:37:26.686.714 [adapter_hccp.cc:988][166235][Get][DeviceIP]hrtGetIfNum success. ifAddrNum[2]. [INFO] HCCP(164046,python):2024-01-10-11:37:26.686.724 [ra_host.c:1930]tid:166235,ra_get_ifaddrs(1930) : Input parameters: phy_id[3], nic_position:[1], interface num[2] [INFO] HCCL(164046,python):2024-01-10-11:37:26.691.121 [adapter_hccp.cc:1018][166235]hrtGetIfAddress: idx[0] ifname[eth3] ip[192.168.103.101%eth3] [INFO] HCCL(164046,python):2024-01-10-11:37:26.691.135 [topoinfo_detect.cc:472][166235]select AF_INET family as device socket family. [INFO] HCCP(164046,python):2024-01-10-11:37:26.691.214 [ra_host.c:825]tid:166235,ra_socket_batch_connect(825) : Input parameters: [0]th, phy_id[3], local_ip[8.92.9.85], remote_ip[8.92.9.85], tag:[topo_detect_default_tag_60000] [INFO] HCCP(164046,python):2024-01-10-11:37:26.693.728 [ra_host.c:863]tid:166235,ra_socket_batch_close(863) : Input parameters: [0]th, phy_id[3], local_ip[8.92.9.85] [INFO] HCCP(164046,python):2024-01-10-11:37:26.737.985 [ra_host.c:525]tid:166235,ra_socket_deinit(525) : Input parameters: phy_id[3] family[2] local_ip[8.92.9.85] [INFO] HCCP(164046,python):2024-01-10-11:37:26.738.007 [rs.c:1257]tid:166235,rs_socket_deinit(1257) : socket deinit success, phy_id:3, local_ip:8.92.9.85 [INFO] HCCP(164046,python):2024-01-10-11:37:26.738.022 [ra_host.c:349]tid:166235,ra_deinit(349) : Input parameters: phy_id[3], nic_position:[0] [INFO] HCCP(164046,python):2024-01-10-11:37:26.918.553 [rs.c:1460]tid:166235,rs_deinit(1460) : rs_deinit chip_id[3] ok [INFO] HCCP(164046,python):2024-01-10-11:37:26.918.572 [ra_host.c:349]tid:166235,ra_deinit(349) : Input parameters: phy_id[3], nic_position:[1] [INFO] HCCP(164046,python):2024-01-10-11:37:26.918.582 [ra_hdc.c:1535]tid:166235,ra_hdc_deinit(1535) : hdc deinit start! phy_id[3] [INFO] HCCP(164046,python):2024-01-10-11:37:26.918.711 [ra_hdc.c:1570]tid:166235,ra_hdc_deinit(1570) : hdc deinit OK! phy_id[3] [INFO] ATRACE(164046,python):2024-01-10-11:37:26.918.948 [atrace_api.c:28](tid:166235) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:26.919.002 [trace_rb_log.c:84](tid:166235) [HCCL_166235_3] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:26.919.015 [atrace_api.c:32](tid:166235) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:26.919.447 [process_mode_manager.cpp:109][OpenProcess][tid:166235] [ProcessModeManager] enter into open process deviceId[3] rankSize[2] [INFO] HCCP(164046,python):2024-01-10-11:37:26.919.466 [ra_host.c:293]tid:166235,ra_init(293) : Input parameters: phy_id[3], nic_position:[1] [INFO] HCCP(164046,python):2024-01-10-11:37:26.919.540 [ra_hdc.c:1465]tid:166235,ra_hdc_init(1465) : hdc init start! logic id is 3, phy id is 3 [INFO] HCCP(164046,python):2024-01-10-11:37:26.919.785 [ra_hdc.c:1500]tid:166235,ra_hdc_init(1500) : hdc init OK! phy_id[3] [INFO] HCCP(164046,python):2024-01-10-11:37:26.922.112 [ra_host.c:389]tid:166235,ra_socket_init(389) : socket init:mode=1 phy_id=3 family=2 ip=3.0.0.0 [INFO] HCCP(164046,python):2024-01-10-11:37:26.922.743 [ra_host.c:903]tid:166235,ra_socket_listen_start(903) : Input parameters: [0]th, phy_id[3], local_ip[3.0.0.0] [INFO] HCCL(164046,python):2024-01-10-11:37:26.928.404 [hccl_impl.cc:430][166235]hccl algorithm: [Module(aiserver)] there are 4 device in level0, using fullmesh algo [TRACE] HCCL(164046,python):2024-01-10-11:37:26.932.964 [status:init] [op_base.cc:481][166235]HcclCommInitRootInfo success,take time [419123]us, rankNum[4], rank[3],rootInfo identifier[8.92.9.85%enp189s0f0_60000_0_1704857845691656], server[8.92.9.85%enp189s0f0], device[3] [WARNING] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:26.933.026 [mindspore/ccsrc/distributed/collective/collective_manager.cc:287] CreateCommunicationGroup] End initialize communication group on the device side: hccl_world_group [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:37:26.933.056 [mindspore/ccsrc/distributed/collective/collective_manager.cc:182] Initialize] End initializing collective communication for backend: Ascend [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:26.933.094 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:285] RecordInitStatus] Status record: system init. [WARNING] ME(164046:281472877868096,MainProcess):2024-01-10-11:37:28.944.497 [mindspore/parallel/_utils.py:259] You are suggested to use mindspore.context.set_auto_parallel_context(parameter_broadcast=True) or mindspore.common.set_seed() to share parameters among multi-devices. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:29.675.658 [mindspore/ccsrc/minddata/dataset/util/task_manager.cc:161] DoServiceStart] Starting Task Manager. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:29.676.378 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at train-labels-idx1-ubyte. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:29.676.410 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at train-images-idx3-ubyte. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:29.676.545 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:29.678.376 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:29.678.512 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:29.678.535 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:29.678.554 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:29.678.589 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:29.678.660 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:29.678.688 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:29.678.703 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:29.678.723 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:29.678.742 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:29.678.776 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:29.678.789 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:29.678.810 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:29.678.840 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:29.679.175 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at train-labels-idx1-ubyte. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:29.679.214 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at train-images-idx3-ubyte. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:30.399.275 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:30.399.406 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:30.445.125 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:30.445.275 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:30.445.298 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:30.445.317 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:30.445.352 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:30.445.425 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:30.445.454 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:30.445.469 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:30.445.491 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:30.445.510 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:30.445.543 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:30.445.581 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:30.445.604 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:30.445.634 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:30.446.062 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at train-labels-idx1-ubyte. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:30.446.090 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at train-images-idx3-ubyte. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.106.151 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.106.288 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.152.235 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.152.393 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.152.417 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.152.436 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.152.472 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.152.546 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.152.576 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.152.617 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.152.640 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.152.661 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.152.695 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.152.709 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.152.731 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.152.760 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.153.112 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] ME(164046:281472877868096,MainProcess):2024-01-10-11:37:31.221.003 [mindspore/dataset/engine/datasets.py:4269] queue_name is newly generated. value is 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.223.974 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.224.208 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.224.387 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.224.431 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.224.471 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.224.545 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.224.576 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.224.591 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.224.613 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.224.632 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.224.665 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.224.679 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.224.701 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.224.730 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.225.109 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:31.225.325 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1729] InitExecDatasetVm] Start InitDataSet Entry [INFO] DEBUG(164046,ffff82e63440,python):2024-01-10-11:37:31.225.646 [mindspore/ccsrc/common/debug/env_config_parser.cc:152] ParseFromFile] The 'env_config_path' in 'mindspore.context.set_context(env_config_path={path})' is empty. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:37:31.225.678 [mindspore/ccsrc/backend/graph_compiler/transform.cc:573] CreateBackend] CreateBackend is: ge [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:31.225.734 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:31.225.755 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEBUG(164046,ffff82e63440,python):2024-01-10-11:37:31.226.006 [mindspore/ccsrc/debug/debugger/debugger.cc:129] Init] Debugger got device_id: 3 [INFO] DEBUG(164046,ffff82e63440,python):2024-01-10-11:37:31.226.032 [mindspore/ccsrc/debug/debugger/debugger.cc:131] Init] Debugger got device_target: Ascend [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:31.335.364 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:433] Initialize] The actor thread number: 5, the kernel thread number: 25 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:31.335.742 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:31.335.767 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:31.335.782 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1252] SetRunMode] Run graph mode with kernel by kernel by configuration. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:37:31.336.398 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:546] CompileGraphs] Status record: start compile function graph: _anonymous__1 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.336.508 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unify_mindir_pm_0_erase_invalid_micro_depend in 9.01 us [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:37:31.337.350 [mindspore/ccsrc/utils/anfalgo.cc:1736] IsNodeOutputDynamicShape] Invalid base shape, node: Default/Return-op0 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:31.337.431 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:31.337.448 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:31.337.482 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:31.337.495 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:31.337.514 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:31.337.527 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:37:31.337.553 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:727] CompileGraph] Compile graph: _anonymous__1, Split segments size: 2 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:37:31.337.616 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:762] CompileGraphFromSegment] Compile normal segment, the first node: @_anonymous__1:CNode_2{[0]: ValueNode InitDataSetQueue} [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:31.337.702 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:421] CompileGraph] Status record: start compile graph. [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:31.337.781 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:1940] ConstructKernelGraph] Create graph: 0 [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:31.338.019 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:2697] ConstructOutput] Output:@_anonymous__1:CNode_2{[0]: ValueNode InitDataSetQueue} [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.338.524 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:193] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:0 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.338.587 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 6.03 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.338.688 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 52.92 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.338.793 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:154] CommonUnifyMindIR] start common unify mindir opt graph:0 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.338.834 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.338.931 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 91.61 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.338.948 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.338.981 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 28.5 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.338.997 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.339.012 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 13.95 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.339.025 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.339.063 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 34.62 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.339.166 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:79] BackendCommonOptimization] Status record: start common optimization. graph id: 0 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.339.208 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: add_dynamic_shape_attr [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.339.250 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_0_add_dynamic_shape_attr in 38.41 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.339.267 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.339.334 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_1_convert_dynamic_broadcast_to in 63.02 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.339.377 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_2_convert_const_input_to_attr in 24.51 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.339.410 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_3_custom_op_const_input_to_attr in 14.15 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.339.440 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 11.44 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.339.507 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 48.87 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.339.526 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 0.79 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.339.585 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_7_flatten_concat_fission in 32.01 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.339.632 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 28.53 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.339.667 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_9_broadcast_to_fusion in 17.12 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.116 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 424.42 us [WARNING] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.217 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:92] BackendCommonOptimization] [PROF]backend_common_optimization costs 1050 usec. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.234 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:99] BackendCommonOptimization] Status record: end common optimization. graph id: 0 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.441 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_0_renorm_split in 53.53 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.463 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.550 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_1_reduce_axis_update in 81.44 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.570 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim HistogramFixedWidth not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.587 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_2_histogram_fixed_width_fusion in 16.08 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.623 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim ClipByNorm not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.638 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_3_clip_by_norm_fission in 36.15 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.690 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.705 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_4_tensor_arry_add_flow_cond1 in 51.24 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.719 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayGather not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.732 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_5_tensor_arry_add_flow_cond2 in 11.71 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.744 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.757 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_6_ge_tensor_arry_cast_index in 11.29 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.778 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArray not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.810 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_7_tensor_array_prepare in 37.96 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.823 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.884 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_8_space_to_batch_nd_attr_update in 57.88 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.900 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.928 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_9_batch_to_space_nd_attr_update in 25.81 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.340.943 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: avg_pool_grad_unify_mindir [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.002 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_10_avg_pool_grad_unify_mindir in 56.18 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.069 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_11_adam_weight_decay_unify_mindir in 46.82 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.136 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_12_cdist_fission in 46.16 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.176 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_13_cdist_grad_fission in 18.88 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.270 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_14_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 73.49 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.327 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_15_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 34.45 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.364 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_16_sparse_softmax_cross_entropy_with_logits_unify_mindir in 18.03 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.401 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_17_dropout_unify_mindir1 in 19.09 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.415 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.460 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_18_dropoutgrad_unify_mindir in 40.23 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.504 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.521 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_19_neighbor_exchange_unify_mindir in 42.39 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.535 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.557 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_20_neighbor_exchange_v2_unify_mindir in 20.46 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.570 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.583 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_21_neighbor_exchange_v2_grad_unify_mindir in 11.7 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.619 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.633 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_22_all_to_all_unify_mindir in 35.85 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.648 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim QuantDTypeCast not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.662 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_23_quant_dtype_cst_adjust in 14.6 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.677 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim FSEDecode not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.700 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_24_fse_decode_adjust in 13.84 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.741 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.755 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_25_bn_split in 39.06 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.768 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.827 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_26_bn_grad_unify_mindir in 55.19 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.843 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.856 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_27_bn_grad_split in 12.6 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.869 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.882 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_28_batchnorm_to_bninfer in 11.34 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.894 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.906 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_29_batchnormgrad_to_bninfergrad in 11.11 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.919 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.341.931 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_30_batch_norm_grad_infer_fission in 11.19 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.342.036 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_31_ascend_mindir_op_adapter in 81.3 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.342.057 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_32_DropoutGenMask in 0.86 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.342.116 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_33_lamb_fission_ge in 42.13 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.342.170 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_34_print_insert_placeholder_for_tensor_name in 36.01 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.342.238 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_35_getnext_for_ge in 48.97 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.342.299 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_36_sync_bn_split in 41.85 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.342.335 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_37_sync_bn_grad_split in 17.74 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.342.404 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_38_adaptive_max_pool2d_ge_fusion in 50.27 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.342.445 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_39_avg_pool_grad_for_ge in 21.05 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.342.467 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:260] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.342.570 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_40_FlashAttentionFusionV1 in 104.35 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.342.588 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:374] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.342.687 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_41_FlashAttentionFusionV1 in 98.03 us [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:31.342.997 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:31.343.022 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:31.343.036 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:132] GetRunMode] RunMode::kKernelMode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.343.247 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 172.87 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.343.442 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_0_seed_adapter in 70.45 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.343.466 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: env_op_attr_update [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.343.564 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_1_env_op_attr_update in 92.84 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.343.623 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_2_process call inline in 23.29 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.343.712 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_3_expander_fallback in 68.22 us [WARNING] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.343.790 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:139] GEBackendOptimizeACL] [PROF]ascend_backend_optimize_acl costs 433 usec. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:31.343.825 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] InitDataSetQueue is not defined in opdef. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.344.249 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 36.12 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.344.325 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 44.72 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.344.381 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 31.3 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.344.450 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 47.36 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.344.489 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_4_insert_type_transform_op in 16.19 us [WARNING] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.344.567 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:179] GEBackendOptimizeACLAfterKernelSelect] [PROF]ascend_backend_optimize_acl_after_kernel_select costs 441 usec. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.344.620 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_0_DropoutGenMask in 0.53 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.344.668 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_1_cse in 27.4 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.344.717 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_redundant_op in 29.33 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.344.744 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_3_eliminate_maketuple_getitem in 7.13 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.344.760 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_4_MergeTransData in 0.59 us [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:31.344.874 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive InitDataSetQueue [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:31.345.006 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive InitDataSetQueue [WARNING] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:31.345.026 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:262] CreateKernel] [PROF]create_kernel costs 161 usec. [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:31.345.080 [mindspore/ccsrc/backend/common/session/session_basic.cc:1140] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 0 start [INFO] DEBUG(164046,ffff82e63440,python):2024-01-10-11:37:31.345.104 [mindspore/ccsrc/debug/summary/summary.cc:33] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 0 start [INFO] DEBUG(164046,ffff82e63440,python):2024-01-10-11:37:31.345.118 [mindspore/ccsrc/debug/summary/summary.cc:54] RecurseSetSummaryNodesForAllGraphs] The total summary nodes is: 0 for graph: 0 [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:31.345.245 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:891] PrintGraphExecuteOrder] Graph 0 execution order: [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:31.345.285 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[0], node name[Default/InitDataSetQueue-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_0:CNode_2{[0]: ValueNode InitDataSetQueue}] [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.345.320 [mindspore/ccsrc/backend/common/somas/somas.cc:143] Assign] Start Somas Assign for graph 0 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.345.350 [mindspore/ccsrc/backend/common/somas/somas.cc:151] Assign] Somas Configure success, configuration info: Device Name: Ascend Run by execution order: 1 Enable debug log: 0 Debug log path: . [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.345.361 [mindspore/ccsrc/backend/common/somas/somas.cc:154] Assign] Start Initialize SOMAS Model [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.345.406 [mindspore/ccsrc/backend/common/somas/somas.cc:1065] GraphOutputProcess] Set 0 graph output tensors' aligned size to 0. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.345.440 [mindspore/ccsrc/backend/common/somas/somas.cc:1135] RefNodeProcess] Special Tensor total size: RefNode: input 0 output 0 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.345.458 [mindspore/ccsrc/backend/common/somas/somas.cc:551] InitSomasModel] No Tensor from graph 0 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.345.469 [mindspore/ccsrc/backend/common/somas/somas.cc:157] Assign] End Initialize SOMAS Model [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:31.345.480 [mindspore/ccsrc/backend/common/somas/somas.cc:160] Assign] No Somas Tensor in graph 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:31.345.491 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:310] DoSomas] Somas allocate success for graph 0 somas size: 0 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:31.345.522 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:801] CreateDeviceAddress] Status record: start create device address. graph id: 0 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:31.345.592 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:808] CreateDeviceAddress] Status record: end create device address. graph id: 0 [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:31.345.621 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1129] CacheGraphOutputToFrontNodeWithIndex] Get graph backend output nodes. [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:31.345.640 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1137] CacheGraphOutputToFrontNodeWithIndex] Get graph front output nodes. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:31.345.657 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:507] CompileGraph] Status record: end compile graph. graph id: 0 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:37:31.345.731 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:805] CompileGraphFromSegment] Compile cut segment, the cut node: @_anonymous__1:CNode_3{[0]: ValueNode Return, [1]: CNode_2} [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:31.345.810 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:521] Transform] Graph(kernel_graph_0) transforms actor begin, strategy:pipeline_with_execution_order [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:31.345.912 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1232] BuildLoopCountActor] Create loop count actor: kernel_graph_0_LoopCountActor [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:31.345.931 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1265] BuildOutputActor] Create output actor: kernel_graph_0_OutputActor [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:31.345.947 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1285] BuildDataPrepareActor] Create data prepare actor: kernel_graph_0_DataPrepareActor [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:31.345.971 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1518] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_0 start. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:31.345.986 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1520] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_0 end. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:31.346.050 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 1_actor_set_kernel_graph_0_memory_actor_insert in 1.3 us [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:31.346.071 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 2_actor_set_kernel_graph_0_invalid_data_arrow_elimination in 1.01999 us [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:31.346.095 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 3_actor_set_kernel_graph_0_multi_actor_fusion in 8.8 us [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:31.346.111 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 4_actor_set_kernel_graph_0_batch_data_arrow_fusion in 0.810003 us [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:31.346.126 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:554] Transform] Graph(kernel_graph_0) transforms actor end. [WARNING] VM(164046,ffff82e63440,python):2024-01-10-11:37:31.346.169 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:612] CompileGraphs] [PROF]compile_func_graph costs 9746 usec. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:37:31.346.188 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:615] CompileGraphs] Status record: end compile function graph: _anonymous__1, produce actor: kernel_graph_0 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:37:31.346.207 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_0 [INFO] GE(164046,python):2024-01-10-11:37:31.346.656 [scalable_config.cc:55][EVENT]167352 ScalableConfig:device total max size: 34359738368, page_mem_size_total_thresold: 32641751449, uncacheable_size_threshold: 17179869184 [INFO] GE(164046,python):2024-01-10-11:37:31.427.439 [graph_var_manager.cc:1424][EVENT]167352 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:31.427.570 [graph_manager.cc:1248][EVENT]167352 PreRun:PreRun start: graph node size 1, session id 1, graph id 0, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:31.428.509 [atrace_api.c:28](tid:167352) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:31.428.588 [trace_rb_log.c:84](tid:167352) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:31.428.603 [atrace_api.c:32](tid:167352) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:31.428.642 [client_manager.cpp:157][SetProfilingCallback][tid:167352] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:31.429.792 [parallel_partitioner.cc:165][EVENT]167352 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [34] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.429.839 [parallel_partitioner.cc:178][EVENT]167352 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.429.891 [graph_prepare.cc:1378][EVENT]167352 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.430.542 [graph_manager.cc:1050][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [673] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.430.569 [graph_manager.cc:1052][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.430.665 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [1] [INFO] GE(164046,python):2024-01-10-11:37:31.430.694 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.430.844 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [139] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.430.858 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.430.980 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [42] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.430.993 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.431.005 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.431.118 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.431.144 [graph_manager.cc:1054][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [562] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.438.607 [graph_manager.cc:1055][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [7447] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.439.504 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:31.439.533 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.439.544 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [20] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.439.554 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [131] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.439.563 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [47] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.439.572 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:31.439.581 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.439.590 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [3] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.439.608 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.440.718 [graph_manager.cc:1056][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2072] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.440.778 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.440.799 [graph_prepare.cc:1982][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [46] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.440.954 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:31.440.973 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.440.983 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.440.992 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [48] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.001 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [5] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.010 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:31.441.019 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.028 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.037 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.105 [graph_prepare.cc:1983][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [292] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.441.128 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.441.140 [graph_prepare.cc:1984][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.441.156 [graph_prepare.cc:1985][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.441.181 [graph_prepare.cc:1986][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.441.194 [graph_prepare.cc:1987][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.441.208 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.441.220 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.441.232 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.441.309 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.323 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.332 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.341 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.350 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.358 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.367 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.375 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.384 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.392 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [0] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.400 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.409 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.417 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.425 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.434 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.442 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.441.464 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.441.479 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.441.505 [graph_prepare.cc:1988][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [301] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.441.517 [graph_manager.cc:1065][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [765] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.454.324 [graph_manager.cc:1077][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12787] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.454.371 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.454.397 [graph_manager.cc:1080][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.175 [graph_manager.cc:1081][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [2755] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.216 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.230 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.242 [graph_manager.cc:1082][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [34] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.269 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.282 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.295 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.421 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [117] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.440 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.452 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.465 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.502 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [27] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.519 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.534 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.552 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.591 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [29] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.605 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.615 [graph_manager.cc:2700][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [351] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.811 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.457.826 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AddNPass is [0] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.457.836 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.457.853 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.457.863 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.457.872 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CastRemovePass is [68] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.457.881 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.457.889 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [33] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.457.898 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [25] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.457.907 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.457.915 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [6] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.457.924 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.457.932 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.457.940 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.457.949 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.457.959 [graph_manager.cc:2741][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [326] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.968 [graph_manager.cc:2752][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.457.990 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.001 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.017 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.031 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.041 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.053 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.092 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [30] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.107 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.120 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.138 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.154 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.166 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.178 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.189 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.198 [graph_manager.cc:2810][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [211] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.221 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.458.232 [graph_manager.cc:2821][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [25] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.260 [graph_manager.cc:1087][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1002] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.395 [graph_manager.cc:1088][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [122] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.449 [graph_manager.cc:1089][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.468 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.482 [graph_manager.cc:1097][EVENT]167352 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:31.458.502 [graph_manager.cc:3325][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.674 [engine_place.cc:144][EVENT]167352 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.688 [engine_place.cc:144][EVENT]167352 Run:The time cost of aicpu_ascend_kernel::CheckSupported is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.768 [graph_manager.cc:3351][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [253] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.784 [graph_manager.cc:3364][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.852 [engine_partitioner.cc:1139][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.868 [engine_partitioner.cc:1142][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.458.979 [engine_partitioner.cc:1148][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [102] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.459.003 [engine_partitioner.cc:1155][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.459.053 [engine_partitioner.cc:1164][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.459.083 [graph_manager.cc:3405][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [286] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.459.101 [graph_manager.cc:3412][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.461.667 [graph_manager.cc:3422][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [2551] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.461.754 [graph_manager.cc:3428][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.461.881 [graph_manager.cc:3467][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [106] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.461.898 [graph_manager.cc:3377][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [3102] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.461.913 [graph_manager.cc:1106][EVENT]167352 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [3417] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.461.925 [graph_manager.cc:1115][EVENT]167352 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:31.461.948 [graph_manager.cc:1130][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.461.978 [graph_manager.cc:1131][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.462.027 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.462.043 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.462.053 [graph_manager.cc:2837][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [60] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.462.100 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.462.112 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.462.121 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondRemovePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.462.130 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of BitcastPass is [0] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.462.138 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.462.147 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:31.462.165 [graph_manager.cc:2864][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [96] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.462.176 [graph_manager.cc:2872][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.462.196 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.462.209 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.462.223 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.462.236 [compile_nodes_pass.cc:88][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.462.246 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.462.274 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.462.308 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [25] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.462.355 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.462.369 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.462.382 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.462.394 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.462.403 [graph_manager.cc:2927][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [209] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.462.450 [graph_manager.cc:2937][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.462.554 [graph_manager.cc:2943][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [93] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.462.576 [graph_manager.cc:2950][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.472.545 [graph_manager.cc:2958][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.472.590 [graph_manager.cc:1132][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [10598] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.472.694 [graph_manager.cc:1135][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [88] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.472.739 [graph_manager.cc:2975][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [25] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.472.924 [graph_manager.cc:2981][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [163] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.472.941 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.472.953 [graph_manager.cc:2986][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.472.963 [graph_manager.cc:1136][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [251] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.473.062 [graph_manager.cc:3555][EVENT]167352 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [59] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.473.156 [engine_partitioner.cc:1139][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.473.171 [engine_partitioner.cc:1142][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.473.240 [engine_partitioner.cc:1148][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [59] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.473.261 [engine_partitioner.cc:1155][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.473.292 [engine_partitioner.cc:1164][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.473.311 [graph_builder.cc:865][EVENT]167352 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [181] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.473.384 [graph_builder.cc:288][EVENT]167352 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [51] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.473.542 [graph_builder.cc:293][EVENT]167352 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [145] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.473.885 [model_builder.cc:1133][EVENT]167352 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [226] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.474.192 [block_mem_assigner.cc:4069][EVENT]167400 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164046,python):2024-01-10-11:37:31.474.224 [block_mem_assigner.cc:4069][EVENT]167399 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164046,python):2024-01-10-11:37:31.474.612 [model_builder.cc:1144][EVENT]167352 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [703] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.474.642 [model_builder.cc:1152][EVENT]167352 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.474.657 [model_builder.cc:1157][EVENT]167352 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.474.814 [model_builder.cc:1167][EVENT]167352 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [146] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.474.834 [model_builder.cc:1174][EVENT]167352 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.474.863 [model_builder.cc:1180][EVENT]167352 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.474.934 [model_builder.cc:1184][EVENT]167352 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [59] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.474.954 [graph_builder.cc:304][EVENT]167352 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [1385] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:31.475.199 [logger.cc:1071] 167352 ModelBindStream: model_id=576, stream_id=1857, flag=0. [INFO] GE(164046,python):2024-01-10-11:37:31.475.281 [task_generator.cc:804][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.475.380 [task_generator.cc:805][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [81] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.476.044 [task_generator.cc:814][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [638] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.476.059 [task_generator.cc:954][EVENT]167352 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [784] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.476.127 [task_generator.cc:967][EVENT]167352 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [37] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:31.476.146 [logger.cc:1084] 167352 ModelUnbindStream: model_id=576, stream_id=1857, [INFO] GE(164046,python):2024-01-10-11:37:31.476.202 [graph_builder.cc:310][EVENT]167352 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [1236] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.476.321 [graph_manager.cc:1152][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3340] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.476.337 [graph_manager.cc:1164][EVENT]167352 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:31.476.376 [graph_manager.cc:1271][EVENT]167352 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [46817] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.476.386 [graph_manager.cc:1272][EVENT]167352 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:37:31.476.694 [atrace_api.c:93](tid:167352) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:37:31.476.717 [atrace_api.c:95](tid:167352) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:37:31.477.387 [model_introduction.cc:236][EVENT]167352 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(164046,python):2024-01-10-11:37:31.477.409 [model_introduction.cc:294][EVENT]167352 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(164046,python):2024-01-10-11:37:31.477.421 [model_introduction.cc:366][EVENT]167352 Data:model io_info size:0 [INFO] GE(164046,python):2024-01-10-11:37:31.480.004 [graph_converter.cc:838][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [772] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.480.069 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.480.325 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [240] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.480.385 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.480.407 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [62] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.480.431 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.480.455 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.480.476 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.480.509 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [24] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.480.552 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.480.561 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [41] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.480.579 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.480.596 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.480.608 [graph_converter.cc:849][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [562] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.480.721 [graph_converter.cc:853][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [104] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.481.173 [graph_converter.cc:857][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [440] micro second. [INFO] GE(164046,python):2024-01-10-11:37:31.481.273 [graph_converter.cc:862][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [77] micro second. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:37:31.482.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_0_LoopCountActor) running, loop count: 1, current count: 1, total running count: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:31.483.173 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_0 execution count: 1, execution time: 136.899 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:37:31.483.230 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:31.483.441 [mindspore/ccsrc/runtime/device/kernel_runtime_manager.cc:35] ClearGraphResource] Clear device Ascend_3 graph 0 runtime resource [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.486.889 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:198] Compile] Input plan: +-Transfer,send_epoch_end:false,total_batch:2340) | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.487.025 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:216] Compile] Plan before optimization: +-Top | +-Transfer,send_epoch_end:false,total_batch:2340) | | +-Repeat(count:1) | | | +-Batch(batch_size:32 drop_remainder:true) | | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.487.064 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:60] PrePass] Running pre pass loops. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.487.087 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.487.124 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.487.212 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.487.241 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.487.256 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.487.284 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.487.297 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/cache_transform_pass.cc:182] RunOnTree] Pre pass: Cache transform pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.487.322 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/cache_transform_pass.cc:199] RunOnTree] Pre pass: Cache transform pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.487.334 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:91] PrePass] Pre pass offload complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.487.349 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:116] PostPass] Running post pass loops. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.487.386 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:135] PostPass] Post passes complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.487.427 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:230] Compile] Plan after optimization: +-Top | +-Transfer,send_epoch_end:false,total_batch:2340) | | +-EpochCtrl(epoch:5) | | | +-Batch(batch_size:32 drop_remainder:true) | | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | | +-MnistDataset [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:31.488.034 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_data_queue.cc:227] AscendTdtQueue] Select MBUF channel, the capacity of data queue is: 128 [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:37:31.488.109 [mindspore/ccsrc/minddata/dataset/engine/datasetops/epoch_ctrl_op.cc:25] EpochCtrlOp] Welcome to Epoch Ctrl Op. [INFO] MD(164046,fffdbeffd0f0,python):2024-01-10-11:37:31.491.061 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at train-labels-idx1-ubyte. [INFO] MD(164046,fffdbeffd0f0,python):2024-01-10-11:37:31.491.113 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at train-images-idx3-ubyte. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:31.496.433 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:456] SendDataToAscend] Device queue, sending data to Ascend. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:31.860.704 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:493] GenerateArgumentsKey] Generate a new compile key for new args, key: 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:31.860.781 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:501] GenerateArgumentsKey] New cached args: [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:31.861.794 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:978] CompileInner] Start compiling, phase: train.1704857851709617664.281469736529552.0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:31.861.832 [mindspore/ccsrc/pipeline/jit/ps/event_message_print.cc:37] PrintEventMessage] Start compiling '_DataWrapper.construct' and it will take a while. Please wait... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:31.872.891 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1659] VmPipeline] This worker is initialized. No need to add worker action. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:37:31.872.977 [mindspore/ccsrc/backend/graph_compiler/transform.cc:573] CreateBackend] CreateBackend is: ge [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:31.873.003 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:31.873.018 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEBUG(164046,ffff82e63440,python):2024-01-10-11:37:31.873.322 [mindspore/ccsrc/debug/debugger/debugger.cc:129] Init] Debugger got device_id: 3 [INFO] DEBUG(164046,ffff82e63440,python):2024-01-10-11:37:31.873.337 [mindspore/ccsrc/debug/debugger/debugger.cc:131] Init] Debugger got device_target: Ascend [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:31.873.361 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1311] Run] Pipeline run [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:31.873.381 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start parse action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:31.885.020 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end parse action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:31.885.083 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start symbol_resolve action. [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.904.774 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] _no_sens_impl_4 update var `grads` with node @_no_sens_impl_4:grads{[0]: CNode_5, [1]: grads} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.905.013 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_no_sens_impl_4] Added global python symbol: {F : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.905.493 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] _no_sens_impl_4 update var `loss` with node @_no_sens_impl_4:loss{[0]: CNode_6, [1]: loss, [2]: CNode_7} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.923.046 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] mindspore_nn_optim_momentum_Momentum_construct_8 update var `gradients` with node @mindspore_nn_optim_momentum_Momentum_construct_8:gradients{[0]: CNode_9, [1]: param_gradients} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.923.356 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] mindspore_nn_optim_momentum_Momentum_construct_8 update var `gradients` with node @mindspore_nn_optim_momentum_Momentum_construct_8:gradients{[0]: CNode_10, [1]: gradients} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.923.612 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] mindspore_nn_optim_momentum_Momentum_construct_8 update var `gradients` with node @mindspore_nn_optim_momentum_Momentum_construct_8:gradients{[0]: CNode_11, [1]: gradients} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.923.880 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] mindspore_nn_optim_momentum_Momentum_construct_8 update var `gradients` with node @mindspore_nn_optim_momentum_Momentum_construct_8:gradients{[0]: CNode_12, [1]: gradients} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.924.601 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_13{[0]: CNode_14, [1]: CNode_15, [2]: CNode_16}, block: 0x4499ba20/mindspore_nn_optim_momentum_Momentum_construct_8, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/nn/optim/momentum.py:211/ self.assignadd(self.global_step, self.global_step_increase_tensor)/ [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.925.177 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_optim_momentum_Momentum_construct_8] Added global python symbol: {F : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.925.422 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_optim_momentum_Momentum_construct_8] Added global python symbol: {_momentum_opt : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.926.214 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_17{[0]: ValueNode Depend, [1]: CNode_18, [2]: CNode_19}, state: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_13{[0]: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_14{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.optim.momentum..', [2]: ValueNode assignadd}, [1]: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_15{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.optim.momentum..', [2]: ValueNode global_step}, [2]: @mindspore_nn_optim_momentum_Momentum_construct_8:CNode_16{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.optim.momentum..', [2]: ValueNode global_step_increase_tensor}} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.940.608 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20] Added global python symbol: {F : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.940.864 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20] Added global python symbol: {_get_datatype : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.941.391 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20] Added global python symbol: {_cast_datatype : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.941.552 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20] Added global python symbol: {mstype : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.941.888 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20 update var `grads` with node @mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20:grads{[0]: CNode_21, [1]: CNode_22, [2]: param_grads} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.942.537 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_wrap_grad_reducer_DistributedGradReducer_construct_20] Added global python symbol: {reduce_opt : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.952.629 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23] Added global python symbol: {_check_is_tensor : Prim[_check_is_tensor]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.953.127 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_24{[0]: CNode_25, [1]: ValueNode logits, [2]: param_logits, [3]: CNode_26}, block: 0x44991630/mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/nn/loss/loss.py:777/ _check_is_tensor('logits', logits, self.cls_name)/ [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.953.652 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_27{[0]: CNode_25, [1]: ValueNode labels, [2]: param_labels, [3]: CNode_28}, block: 0x44991630/mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/nn/loss/loss.py:778/ _check_is_tensor('labels', labels, self.cls_name)/ [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.954.287 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_29{[0]: ValueNode Depend, [1]: CNode_30, [2]: CNode_31}, state: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_32{[0]: ValueNode MakeTuple, [1]: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_24{[0]: CNode_25, [1]: ValueNode logits, [2]: param_logits, [3]: CNode_26}, [2]: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_23:CNode_27{[0]: CNode_25, [1]: ValueNode labels, [2]: param_labels, [3]: CNode_28}} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.963.384 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_34, [1]: param_x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.963.665 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_35, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.963.946 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_36, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.964.205 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_37, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.964.459 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_38, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.964.711 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_39, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.964.976 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_40, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.965.239 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_41, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.965.503 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_42, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.965.823 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_43, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.966.086 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_44, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.966.346 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_33 update var `x` with node @__main___LeNet5_construct_33:x{[0]: CNode_45, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.977.885 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_46] Added global python symbol: {len : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.978.051 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.978.396 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.978.845 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.979.366 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_46:CNode_48{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.979.505 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_47:x{[0]: CNode_49, [1]: param_фx, [2]: CNode_48} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.979.981 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_46:CNode_50{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.980.422 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_47] Added global python symbol: {len : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.980.491 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_46:CNode_51{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.980.592 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_46:x_shape{[0]: CNode_52, [1]: param_x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.980.725 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.981.025 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.981.288 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_47] Added global python symbol: {F : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.981.381 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_46] Added global python symbol: {F : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.981.433 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_47 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_46:CNode_53{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.981.797 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.988.254 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_54] Added global python symbol: {len : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.988.418 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.988.735 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.988.900 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.989.352 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_54:CNode_56{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.989.504 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_55:x{[0]: CNode_57, [1]: param_фx, [2]: CNode_56} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.989.962 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_54:CNode_58{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.990.404 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_55] Added global python symbol: {len : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.990.473 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_54:CNode_59{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.990.576 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_54:x_shape{[0]: CNode_60, [1]: param_x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.990.713 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.991.004 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.991.290 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_55] Added global python symbol: {F : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.991.396 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_54] Added global python symbol: {F : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.991.451 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_55 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_54:CNode_61{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.991.777 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.995.446 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_62] Added global python symbol: {len : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.995.606 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.995.922 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.996.098 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.996.538 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_62:CNode_64{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.996.676 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_63:x{[0]: CNode_65, [1]: param_фx, [2]: CNode_64} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.997.114 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_62:CNode_66{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.997.557 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_63] Added global python symbol: {len : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.997.624 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_62:CNode_67{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.997.774 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_62:x_shape{[0]: CNode_68, [1]: param_x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.997.915 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.998.195 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.998.458 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_63] Added global python symbol: {F : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.998.551 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_62] Added global python symbol: {F : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.998.601 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_63 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_62:CNode_69{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:31.998.902 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.002.719 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Flatten_construct_70] Added global python symbol: {F : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.016.317 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1261] ParseNameConstant] The NameConstant is bool:False [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.016.617 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:3 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.016.878 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.017.003 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1261] ParseNameConstant] The NameConstant is bool:True [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.017.753 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_pooling_MaxPool2d_construct_71] Added global python symbol: {isinstance : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.017.855 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_pooling_MaxPool2d_construct_72] Added global python symbol: {isinstance : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.017.918 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_pooling_MaxPool2d_construct_71 update var `isinstance` with node @mindspore_nn_layer_pooling_MaxPool2d_construct_72:CNode_73{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.pooling', [2]: ValueNode isinstance} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.018.079 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_pooling_MaxPool2d_construct_71] Added global python symbol: {tuple : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.018.158 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_pooling_MaxPool2d_construct_72] Added global python symbol: {tuple : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.018.206 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_pooling_MaxPool2d_construct_71 update var `tuple` with node @mindspore_nn_layer_pooling_MaxPool2d_construct_72:CNode_74{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.pooling', [2]: ValueNode tuple} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.018.567 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.018.683 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.018.901 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.019.010 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.019.284 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.060.711 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.060.885 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.061.580 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @canonicalize_axis_75:CNode_76{[0]: ValueNode check_axis_valid_77, [1]: param_axis, [2]: ndim}, block: 0x4aa1bec0/canonicalize_axis_75, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1606/ check_axis_valid(axis, ndim)/ [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.061.837 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.062.126 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @canonicalize_axis_75:CNode_78{[0]: ValueNode Depend, [1]: CNode_79, [2]: CNode_80}, state: @canonicalize_axis_75:CNode_76{[0]: ValueNode check_axis_valid_77, [1]: @canonicalize_axis_75:param_axis, [2]: @canonicalize_axis_75:ndim{[0]: CNode_81}} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.062.405 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {isinstance : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.062.553 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {Tensor : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.063.117 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {int : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.063.557 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {bool : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.064.243 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {check_flatten_order_const : Prim[check_flatten_order]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.064.840 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @2↓flatten_83:CNode_84{[0]: CNode_85, [1]: param_order}, block: 0x4aa43660/2↓flatten_83, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1615/ check_flatten_order_const(order)/ [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.065.255 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {rank_ : Prim[Rank]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.065.598 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.065.658 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.065.930 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {reshape_ : Prim[Reshape]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.066.110 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.066.419 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {ops : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.066.629 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.067.127 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {transpose_ : Prim[Transpose]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.067.541 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_86] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.067.649 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.067.713 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `shape_` with node @flatten_82:CNode_87{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode shape_} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.068.039 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_86] Added global python symbol: {rank_ : Prim[Rank]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.068.107 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `rank_` with node @flatten_82:CNode_88{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode rank_} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.068.396 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `start_dim` with node @flatten_82:param_start_dim [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.068.541 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.068.689 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `end_dim` with node @flatten_82:param_end_dim [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.068.801 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.069.083 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.069.139 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.069.373 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_86] Added global python symbol: {reshape_ : Prim[Reshape]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.069.440 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `reshape_` with node @flatten_82:CNode_89{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode reshape_} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.069.634 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.070.002 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_86] Added global python symbol: {flatten_ : Prim[Flatten]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.070.114 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_82] Added global python symbol: {flatten_ : Prim[Flatten]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.070.196 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `flatten_` with node @flatten_82:CNode_90{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode flatten_} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.070.544 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `canonicalize_axis` with node ValueNode canonicalize_axis_75 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.070.987 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_86 update var `check_dim_valid` with node ValueNode check_dim_valid_91 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.071.587 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @4↓flatten_92:CNode_93{[0]: ValueNode check_dim_valid_91, [1]: start_dim, [2]: end_dim}, block: 0x4aa6f920/4↓flatten_92, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1636/ check_dim_valid(start_dim, end_dim)/ [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.071.829 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.071.886 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.072.139 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.072.566 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.073.079 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.073.626 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.074.148 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @2↓flatten_83:CNode_94{[0]: ValueNode Depend, [1]: CNode_95, [2]: CNode_96}, state: @2↓flatten_83:CNode_84{[0]: @flatten_82:CNode_85{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode check_flatten_order_const}, [1]: @flatten_82:param_order} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.074.262 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @4↓flatten_92:CNode_97{[0]: ValueNode Depend, [1]: CNode_98, [2]: CNode_99}, state: @4↓flatten_92:CNode_93{[0]: ValueNode check_dim_valid_91, [1]: @4↓flatten_92:idx{[0]: ValueNode canonicalize_axis_75, [1]: param_start_dim, [2]: x_rank}, [2]: @4↓flatten_92:end_dim{[0]: ValueNode canonicalize_axis_75, [1]: param_end_dim, [2]: x_rank}} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.074.370 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.074.463 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.075.690 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:174] CheckFuncReturn] Function must has 'return' statement, but missing in function 'flatten' at /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1557. FuncGraph: ↓check_dim_valid_100. We will add a 'return None' statement automatically. [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.075.803 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:174] CheckFuncReturn] Function must has 'return' statement, but missing in function 'flatten' at /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1557. FuncGraph: ↓check_axis_valid_101. We will add a 'return None' statement automatically. [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.119.400 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [shape_102] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.127.840 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end symbol_resolve action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.127.897 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start graph_reusing action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.127.916 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.basic.DenseDense[True, None]_ID [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.127.929 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.conv.Conv2dConv2dodict_values([6, 16, 5, 1, 'valid', 0, False, ])_ID [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.127.940 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.conv.Conv2dConv2dodict_values([1, 6, 5, 1, 'valid', 0, False, ])_ID [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.127.956 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end graph_reusing action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.127.974 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start meta_unpack_prepare action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.128.999 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end meta_unpack_prepare action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.129.043 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pre_cconv action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.129.060 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pre_cconv action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.129.078 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start abstract_specialize action. [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.151.651 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_106{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.151.728 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.152.859 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_108{[0]: CNode_109}, new_node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_110{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.152.916 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_108{[0]: CNode_109}, new node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_108{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.155.937 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_equal_scalar_112] Added global python symbol: {F : } [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.156.309 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 4, Shape: NoShape), AbstractScalar(Type: Int64, Value: 3, Shape: NoShape)}, g: _equal_scalar_112 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.157.110 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_113:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_113:CNode_115{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.157.176 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_113:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_113:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.160.634 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_117{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.160.699 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.161.030 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_118{[0]: CNode_119}, new_node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_120{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.161.085 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_118{[0]: CNode_119}, new node: @mindspore_nn_layer_pooling_MaxPool2d_construct_107:CNode_118{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.161.746 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_121:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_121:CNode_122{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.161.813 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_121:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_121:CNode_114{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.169.226 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_logical_not_scala_124] Added global python symbol: {auto_generate : } [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.169.670 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Bool, Value: true, Shape: NoShape)}, g: _logical_not_scala_124 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.175.165 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bool_not_125] Added global python symbol: {_get_cache_prim : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.175.312 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bool_not_125] Added global python symbol: {BoolNot : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.189.401 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_126] Added global python symbol: {str : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.189.871 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↻_get_cache_prim_for_pynative_127] Added global python symbol: {str : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.190.140 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↻_get_cache_prim_for_pynative_127 update var `str` with node @↵_get_cache_prim_for_pynative_128:param_фstr [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.190.777 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_126] Added global python symbol: {tuple : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.190.975 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] _get_cache_prim_for_pynative_129 update var `key` with node @_get_cache_prim_for_pynative_129:key{[0]: CNode_130, [1]: key, [2]: CNode_131} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.191.752 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_132] Added global python symbol: {str : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.192.345 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_132] Added global python symbol: {_PRIM_CACHE : {}} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.192.443 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_126] Added global python symbol: {_PRIM_CACHE : {}} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.192.699 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_132] Added global python symbol: {Primitive : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.192.789 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_126] Added global python symbol: {Primitive : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.193.490 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @✓↓_get_cache_prim_for_pynative_133:CNode_134{[0]: ValueNode MetaFuncGraph-unpack_call.135, [1]: CNode_136, [2]: param_фargs, [3]: param_фkwargs}, block: 0x4a062cd0/✓↓_get_cache_prim_for_pynative_133, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/_primitive_cache.py:84/ prim.__init__(*args, **kwargs)/ [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.194.124 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 2↓_get_cache_prim_for_pynative_137 update var `key` with node @↓_get_cache_prim_for_pynative_138:key{[0]: param_фstr, [1]: param_фkey} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.194.278 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @↻_get_cache_prim_for_pynative_139:CNode_140{[0]: ValueNode Depend, [1]: CNode_141, [2]: CNode_142}, state: @↻_get_cache_prim_for_pynative_139:CNode_143{[0]: ValueNode MetaFuncGraph-add.144, [1]: @↵_get_cache_prim_for_pynative_132:param_@CNode_143, [2]: ValueNode 1} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.194.379 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @✓↓_get_cache_prim_for_pynative_133:CNode_145{[0]: ValueNode Depend, [1]: CNode_146, [2]: CNode_147}, state: @✓↓_get_cache_prim_for_pynative_133:CNode_134{[0]: ValueNode MetaFuncGraph-unpack_call.135, [1]: @✓↓_get_cache_prim_for_pynative_133:CNode_136{[0]: ValueNode getattr, [1]: prim, [2]: ValueNode __init__}, [2]: @↵_get_cache_prim_for_pynative_132:param_фargs, [3]: @↵_get_cache_prim_for_pynative_132:param_фkwargs} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.195.680 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_148:CNode_149{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.195.746 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_148:CNode_150{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.195.788 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_148:CNode_151{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.196.544 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BoolNot. node: @bool_not_125:CNode_152{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x}, new_node: @bool_not_125:CNode_153{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.196.600 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BoolNot. node: @bool_not_125:CNode_152{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x}, new node: @bool_not_125:CNode_152{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.202.599 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_equal_string_154] Added global python symbol: {F : } [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.202.966 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: String, Value: C, Shape: NoShape), AbstractScalar(Type: String, Value: F, Shape: NoShape)}, g: _equal_string_154 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.204.521 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_155:CNode_156{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_155:CNode_157{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.204.601 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_155:CNode_156{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_155:CNode_156{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.207.225 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_neg_scalar_159] Added global python symbol: {F : } [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.207.614 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 1, Shape: NoShape)}, g: _neg_scalar_159 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.208.296 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarUsub. node: @_neg_scalar_160:CNode_161{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x}, new_node: @_neg_scalar_160:CNode_162{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.208.357 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarUsub. node: @_neg_scalar_160:CNode_161{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x}, new node: @_neg_scalar_160:CNode_161{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.209.057 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_163:CNode_164{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_163:CNode_165{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.209.125 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_163:CNode_164{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_163:CNode_164{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.209.569 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @3↓flatten_166:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput}, new_node: @3↓flatten_166:CNode_167{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.209.636 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @3↓flatten_166:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput}, new node: @3↓flatten_166:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.213.932 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_169] Added global python symbol: {F : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.214.654 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_169] Added global python symbol: {InSequence : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.215.062 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_169] Added global python symbol: {const_utils : } [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.215.580 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 4, Shape: NoShape), AbstractTuple{ element[0]: AbstractScalar(Type: Int64, Value: 0, Shape: NoShape), element[1]: AbstractScalar(Type: Int64, Value: 1, Shape: NoShape), sequence_nodes: {@✓3↓flatten_170:CNode_171{[0]: ValueNode MakeTuple, [1]: ValueNode 0, [2]: ValueNode 1}, elements_use_flags: {ptr: 0x4a0c97b0, value: [const vector]{0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: _number_in_tuple_169 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.219.291 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Flatten. node: @↓✓3↓flatten_172:CNode_173{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput}, new_node: @↓✓3↓flatten_172:CNode_174{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.219.364 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Flatten. node: @↓✓3↓flatten_172:CNode_173{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput}, new node: @↓✓3↓flatten_172:CNode_173{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.219.436 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:502] SendDataToAscend] Begin to send data to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.219.506 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:1182] PrintBeginInfoWhenFirstBatch] Loading dataset and begin to push first batch into device ... [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.219.696 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_175:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_175:CNode_176{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.219.751 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_175:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_175:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.221.121 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:1192] PrintEndInfoWhenFirstBatch] Loading dataset and push first batch into device successful. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.221.148 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.221.598 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.222.107 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 3 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.222.731 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_not_equal_scalar_178] Added global python symbol: {F : } [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.223.103 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 2, Shape: NoShape), AbstractScalar(Type: Int64, Value: 2, Shape: NoShape)}, g: _not_equal_scalar_178 [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.223.190 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 4 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.223.910 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_179:CNode_180{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_179:CNode_181{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.223.978 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_179:CNode_180{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_179:CNode_180{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.224.308 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 5 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.225.396 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 6 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.226.038 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_183:CNode_184{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_183:CNode_185{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.226.110 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_183:CNode_184{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_183:CNode_184{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.226.729 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 7 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.227.325 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_186:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_186:CNode_187{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias, [3]: ValueNode 0} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.227.410 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_186:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_186:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias, [3]: ValueNode 0} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.227.644 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 8 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.227.750 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_188{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.227.806 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.227.990 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_189:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_189:CNode_190{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.228.039 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_189:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_189:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.228.621 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 9 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.228.896 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_191:CNode_192{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_191:CNode_193{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.228.963 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_191:CNode_192{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_191:CNode_192{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.229.720 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 10 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.230.911 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 11 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.230.965 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_194:CNode_195{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_194:CNode_196{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.231.038 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_194:CNode_195{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_194:CNode_195{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.232.022 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 12 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.232.251 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_197:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_197:CNode_198{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias, [3]: ValueNode 0} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.232.324 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_197:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_197:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias, [3]: ValueNode 0} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.232.586 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_199{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.232.639 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_104:CNode_105{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.232.825 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_200:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_200:CNode_201{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.232.878 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_200:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_200:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.233.082 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 13 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.233.792 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_202:CNode_203{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_202:CNode_204{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.233.859 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_202:CNode_203{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_202:CNode_203{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.234.366 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 14 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.235.305 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 15 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.235.717 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_205:CNode_206{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_205:CNode_207{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.235.784 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_205:CNode_206{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_205:CNode_206{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.236.378 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 16 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.237.009 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_208:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_208:CNode_209{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias, [3]: ValueNode 0} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.237.081 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_208:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_208:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias, [3]: ValueNode 0} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.237.495 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 17 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.238.658 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 18 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.239.715 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 19 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.239.980 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny)}, g: hyper_map_212 [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.240.897 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 20 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.242.140 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 21 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.242.471 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_ones_like_tensor_213] Added global python symbol: {P : } [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.242.832 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny)}, g: _ones_like_tensor_213 [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.243.165 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 22 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.243.968 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: OnesLike. node: @_ones_like_tensor_214:grads{[0]: ValueNode PrimFunc_OnesLike, [1]: param_x}, new_node: @_ones_like_tensor_214:CNode_215{[0]: ValueNode PrimFunc_OnesLike, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.244.033 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_OnesLike. node: @_ones_like_tensor_214:grads{[0]: ValueNode PrimFunc_OnesLike, [1]: param_x}, new node: @_ones_like_tensor_214:grads{[0]: ValueNode PrimFunc_OnesLike, [1]: param_x} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.244.255 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 23 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.244.414 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: env_get(AbstractScalar(Type: Object:EnvType, Value: ValueAny, Shape: NoShape), )}, AbstractTuple{ element[0]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[1]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[2]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[3]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[4]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[5]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[6]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[7]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), sequence_nodes: {@_no_sens_impl_216:CNode_217{[0]: ValueNode MakeTuple, [1]: param_conv1.weight, [2]: param_conv2.weight, [3]: param_fc1.weight, [4]: param_fc1.bias, [5]: param_fc2.weight, [6]: param_fc2.bias, [7]: param_fc3.weight, [8]: param_fc3.bias}, elements_use_flags: {ptr: 0x4a1b8f40, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: hyper_map_218 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.244.777 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: env_get(AbstractScalar(Type: Object:EnvType, Value: ValueAny, Shape: NoShape), )}, AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny)}, g: hyper_map_219 [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.245.357 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 24 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.246.610 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 25 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.247.658 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 26 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.247.671 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensor_env_get_221] Added global python symbol: {environ_get : Prim[EnvironGet]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.247.910 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensor_env_get_221] Added global python symbol: {ref_to_embed : Prim[RefToEmbed]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.248.174 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensor_env_get_221] Added global python symbol: {tensor_zeros_like : Prim[ZerosLike]} [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.248.484 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Object:EnvType, Value: ValueAny, Shape: NoShape), AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny)}, g: _tensor_env_get_221 [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.248.797 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 27 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.249.514 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_222:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_222:CNode_223{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.249.589 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_222:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_222:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.249.986 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 28 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.250.789 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_224:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_224:CNode_225{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.250.855 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_224:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_224:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.250.988 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 29 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.251.942 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_226:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_226:CNode_227{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.252.010 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_226:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_226:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.252.246 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 30 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.253.081 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_228:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_228:CNode_229{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.253.148 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_228:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_228:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.254.347 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_230:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_230:CNode_231{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.254.430 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_230:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_230:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.255.463 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 31 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.255.503 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_232:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_232:CNode_233{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.255.567 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_232:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_232:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.255.902 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 32 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.256.512 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 33 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.256.638 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_234:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_234:CNode_235{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.256.703 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_234:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_234:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.257.790 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ZerosLike. node: @_tensor_env_get_236:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new_node: @_tensor_env_get_236:CNode_237{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.257.839 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 34 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.257.858 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ZerosLike. node: @_tensor_env_get_236:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter}, new node: @_tensor_env_get_236:grads{[0]: ValueNode PrimFunc_ZerosLike, [1]: param_parameter} [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.258.794 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: S-signature()}, AbstractTuple{ element[0]: AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[1]: AbstractTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[2]: AbstractTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[3]: AbstractTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[4]: AbstractTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[5]: AbstractTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[6]: AbstractTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[7]: AbstractTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), sequence_nodes: {@hyper_map_239:grads{[0]: ValueNode MakeTuple, [1]: grads, [2]: grads, [3]: grads, [4]: grads, [5]: grads, [6]: grads, [7]: grads, [8]: grads}, elements_use_flags: {ptr: 0x4a2544d0, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: map_240 [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.259.237 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 35 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.260.542 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 36 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.261.775 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensors_get_datatype_242] Added global python symbol: {F : } [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.261.843 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 37 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.262.126 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny)}, g: _tensors_get_datatype_242 [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.263.292 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 38 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.264.663 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 39 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.266.051 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 40 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.267.362 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 41 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.268.713 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 42 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.270.190 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 43 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.271.202 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensors_cast_datatype_244] Added global python symbol: {F : } [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.271.410 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 44 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.271.567 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractType(Value: Float32), AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny)}, g: _tensors_cast_datatype_244 [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.272.675 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 45 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.273.867 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 46 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.275.562 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 47 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.277.022 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 48 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.278.784 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 49 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.279.338 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: S-signature(AbstractTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x47018f80, value: Tensor(shape=[], dtype=Float32, value=0.25)), AbstractScalar(Type: Bool, Value: true, Shape: NoShape), Prim: S_Prim_AllGather, Prim: S_Prim_AllReduce, )}, AbstractTuple{ element[0]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[1]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[2]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[3]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[4]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[5]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[6]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), element[7]: AbstractScalar(Type: Bool, Value: true, Shape: NoShape), sequence_nodes: {ValueNode (true, true, true, true, true, true, true, true), elements_use_flags: {ptr: 0x4a341780, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} , AbstractTuple{ element[0]: AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[1]: AbstractTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[2]: AbstractTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[3]: AbstractTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[4]: AbstractTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[5]: AbstractTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[6]: AbstractTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[7]: AbstractTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), sequence_nodes: {@map_245:grads{[0]: ValueNode MakeTuple, [1]: grads, [2]: grads, [3]: grads, [4]: grads, [5]: grads, [6]: grads, [7]: grads, [8]: grads}, elements_use_flags: {ptr: 0x4a320100, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: map_246 [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.280.492 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 50 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.282.161 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 51 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.283.820 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 52 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.285.416 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 53 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.286.063 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensors_allreduce_248] Added global python symbol: {F : } [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.287.079 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 54 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.287.114 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x47018f80, value: Tensor(shape=[], dtype=Float32, value=0.25)), AbstractScalar(Type: Bool, Value: true, Shape: NoShape), Prim: S_Prim_AllGather, Prim: S_Prim_AllReduce, AbstractScalar(Type: Bool, Value: true, Shape: NoShape), AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny)}, g: _tensors_allreduce_248 [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.288.752 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 55 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.290.104 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_249:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_249:CNode_250{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.290.183 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_249:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_249:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.290.523 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 56 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.292.134 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 57 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.292.511 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_251:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_251:CNode_252{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.292.591 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_251:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_251:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.293.876 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 58 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.294.845 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_253:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_253:CNode_254{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.294.930 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_253:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_253:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.295.504 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 59 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.297.226 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 60 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.297.525 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_255:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_255:CNode_256{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.297.633 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_255:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_255:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.298.838 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 61 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.299.889 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_257:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_257:CNode_258{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.299.968 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_257:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_257:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.300.513 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 62 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.302.303 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_259:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_259:CNode_260{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.302.378 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 63 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.302.384 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_259:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_259:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.303.881 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 64 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.304.914 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_261:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_261:CNode_262{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.304.996 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_261:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_261:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.305.496 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 65 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.307.386 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 66 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.307.521 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Mul. node: @2✓_tensors_allreduce_263:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new_node: @2✓_tensors_allreduce_263:CNode_264{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.307.599 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Mul. node: @2✓_tensors_allreduce_263:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad}, new node: @2✓_tensors_allreduce_263:new_grad{[0]: ValueNode PrimFunc_Mul, [1]: new_grad, [2]: new_grad} [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.308.330 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: S-signature()}, AbstractTuple{ element[0]: AbstractType(Value: Float32), element[1]: AbstractType(Value: Float32), element[2]: AbstractType(Value: Float32), element[3]: AbstractType(Value: Float32), element[4]: AbstractType(Value: Float32), element[5]: AbstractType(Value: Float32), element[6]: AbstractType(Value: Float32), element[7]: AbstractType(Value: Float32), sequence_nodes: {@map_265:datatypes{[0]: ValueNode MakeTuple, [1]: datatypes, [2]: datatypes, [3]: datatypes, [4]: datatypes, [5]: datatypes, [6]: datatypes, [7]: datatypes, [8]: datatypes}, elements_use_flags: {ptr: 0x4a2dc970, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} , AbstractTuple{ element[0]: AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[1]: AbstractTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[2]: AbstractTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[3]: AbstractTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[4]: AbstractTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[5]: AbstractTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[6]: AbstractTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[7]: AbstractTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), sequence_nodes: {@map_266:new_grad{[0]: ValueNode MakeTuple, [1]: new_grad, [2]: new_grad, [3]: new_grad, [4]: new_grad, [5]: new_grad, [6]: new_grad, [7]: new_grad, [8]: new_grad}, elements_use_flags: {ptr: 0x4da83e00, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: map_267 [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.309.115 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 67 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.311.014 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 68 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.312.659 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 69 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.314.453 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 70 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.315.869 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 71 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.317.456 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 72 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.318.162 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: S-signature(Prim: S_Prim_ApplyMomentum, AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), )}, AbstractTuple{ element[0]: AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[1]: AbstractTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[2]: AbstractTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[3]: AbstractTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[4]: AbstractTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[5]: AbstractTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[6]: AbstractTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), element[7]: AbstractTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), sequence_nodes: {@map_270:new_grad{[0]: ValueNode MakeTuple, [1]: new_grad, [2]: new_grad, [3]: new_grad, [4]: new_grad, [5]: new_grad, [6]: new_grad, [7]: new_grad, [8]: new_grad}, elements_use_flags: {ptr: 0x4db061e0, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} , AbstractTuple{ element[0]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[1]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[2]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[3]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[4]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[5]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[6]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[7]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), sequence_nodes: {@mindspore_nn_optim_momentum_Momentum_construct_271:CNode_272{[0]: ValueNode MakeTuple, [1]: param_conv1.weight, [2]: param_conv2.weight, [3]: param_fc1.weight, [4]: param_fc1.bias, [5]: param_fc2.weight, [6]: param_fc2.bias, [7]: param_fc3.weight, [8]: param_fc3.bias}, elements_use_flags: {ptr: 0x4db1ba30, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} , AbstractTuple{ element[0]: AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[1]: AbstractRefTensor(key: moments.conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[2]: AbstractRefTensor(key: moments.fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[3]: AbstractRefTensor(key: moments.fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[4]: AbstractRefTensor(key: moments.fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[5]: AbstractRefTensor(key: moments.fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[6]: AbstractRefTensor(key: moments.fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), element[7]: AbstractRefTensor(key: moments.fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), sequence_nodes: {@mindspore_nn_optim_momentum_Momentum_construct_271:CNode_273{[0]: ValueNode MakeTuple, [1]: param_moments.conv1.weight, [2]: param_moments.conv2.weight, [3]: param_moments.fc1.weight, [4]: param_moments.fc1.bias, [5]: param_moments.fc2.weight, [6]: param_moments.fc2.bias, [7]: param_moments.fc3.weight, [8]: param_moments.fc3.bias}, elements_use_flags: {ptr: 0x4db1c2d0, value: [const vector]{0, 0, 0, 0, 0, 0, 0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: hyper_map_274 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.318.815 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{PartialAbstractClosure{MetaFuncGraphAbstractClosure: S-signature(Prim: S_Prim_ApplyMomentum, AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), )}, AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny)}, g: hyper_map_275 [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.319.136 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 73 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.320.670 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 74 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.322.472 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 75 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.323.054 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_tensor_run_opt_ext_277] Added global python symbol: {F : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.323.277 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1261] ParseNameConstant] The NameConstant is bool:True [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.323.755 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{Prim: S_Prim_ApplyMomentum, AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), AbstractTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny)}, g: _tensor_run_opt_ext_277 [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.323.934 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 76 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.325.559 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 77 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.327.130 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 78 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.328.781 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 79 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.330.499 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 80 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.332.017 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 81 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.332.375 [mindspore/ccsrc/pipeline/jit/ps/action.cc:282] AbstractAnalyze] function call depth: 0, simulate call depth: 0 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.332.588 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:222] Run] Specialize set top func graph context: {FuncGraph: mindspore_train_dataset_helper__DataWrapper_construct_103 Args: [0]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [1]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [2]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [3]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [4]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [5]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [6]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [7]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [8]: AbstractRefTensor(key: global_step ref_value: AbstractRefTensor(shape: (1), element: AbstractScalar(Type: Int32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [9]: AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [10]: AbstractRefTensor(key: moments.conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [11]: AbstractRefTensor(key: moments.fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [12]: AbstractRefTensor(key: moments.fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [13]: AbstractRefTensor(key: moments.fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [14]: AbstractRefTensor(key: moments.fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [15]: AbstractRefTensor(key: moments.fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [16]: AbstractRefTensor(key: moments.fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [17]: AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [18]: AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), Parent: } [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.332.782 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @mindspore_train_dataset_helper__DataWrapper_construct_103:CNode_278{[0]: ValueNode MetaFuncGraph-unpack_call.279, [1]: ValueNode mindspore_nn_wrap_cell_wrapper_TrainOneStepCell_construct_280, [2]: outputs}, flag: 1 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.333.135 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @UnpackCall_281:CNode_278{[0]: param_282, [1]: CNode_278, [2]: CNode_278}, flag: 1 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.333.501 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @mindspore_nn_wrap_cell_wrapper_TrainOneStepCell_construct_283:CNode_284{[0]: ValueNode ✓mindspore_nn_wrap_cell_wrapper_TrainOneStepCell_construct_285}, flag: 1 [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.333.638 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 82 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.333.745 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @✓mindspore_nn_wrap_cell_wrapper_TrainOneStepCell_construct_285:CNode_286{[0]: ValueNode MetaFuncGraph-unpack_call.287, [1]: ValueNode _no_sens_impl_288, [2]: CNode_289}, flag: 1 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.334.003 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @UnpackCall_290:CNode_286{[0]: param_291, [1]: CNode_286, [2]: CNode_286}, flag: 1 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.334.170 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @_no_sens_impl_216:CNode_292{[0]: ValueNode ✗_no_sens_impl_293}, flag: 1 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.334.275 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @✗_no_sens_impl_293:CNode_294{[0]: ValueNode ↓_no_sens_impl_295}, flag: 1 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.334.354 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @_no_sens_impl_216:loss{[0]: ValueNode S_Prim_Depend, [1]: loss, [2]: CNode_7}, flag: 1 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.334.406 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @_no_sens_impl_216:CNode_7{[0]: ValueNode mindspore_nn_optim_momentum_Momentum_construct_271, [1]: grads}, flag: 1 [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.335.254 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 83 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.336.923 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 84 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.338.576 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 85 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.340.238 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 86 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.340.370 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:1046] ProcessNode] Don't build value node for CNode which contains isolated side-effect inputs, node: @mindspore_nn_optim_momentum_Momentum_construct_271:CNode_17{[0]: ValueNode Depend, [1]: CNode_18, [2]: CNode_19}, flag: 1 [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.341.821 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 87 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.343.468 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 88 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.345.026 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 89 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.346.732 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 90 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.348.320 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 91 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.350.039 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 92 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.351.630 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 93 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.353.309 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 94 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.354.933 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 95 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.356.583 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 96 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.358.346 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 97 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.359.809 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 98 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.361.511 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 99 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.363.238 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 100 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.367.995 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end abstract_specialize action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.368.060 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pack_expand action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.368.491 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pack_expand action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.368.535 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start auto_monad action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.371.757 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end auto_monad action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.371.808 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start inline action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.371.829 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end inline action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.371.850 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pre_auto_parallel action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.371.895 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pre_auto_parallel action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.371.918 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pipeline_split action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.371.934 [mindspore/ccsrc/pipeline/jit/ps/pipeline_split.cc:247] PipelineSplit] Only auto_parallel and semi_auto_parallel support pipeline split. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.371.946 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pipeline_split action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.371.965 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start optimize action. [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.382.610 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [fill_296] Added global python symbol: {cast_ : Prim[Cast]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.382.855 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] fill_296 update var `value` with node @fill_296:value{[0]: CNode_297, [1]: param_value, [2]: param_type} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.383.103 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [fill_296] Added global python symbol: {fillv2_ : Prim[FillV2]} [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:37:32.406.683 [mindspore/ccsrc/frontend/parallel/pynative_shard/pynative_shard.cc:334] PynativeShard] Only auto_parallel and semi_auto_parallel support pynative shard [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:37:32.406.769 [mindspore/ccsrc/frontend/parallel/step_parallel.cc:3009] StepParallel] Strategies would be ignored in data_parallel, shard() only valid in [semi_]auto_parallel. [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.425.136 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bprop_depend_298] Added global python symbol: {C : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:37:32.430.212 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bprop_load_299] Added global python symbol: {C : } [INFO] OPTIMIZER(164046,ffff82e63440,python):2024-01-10-11:37:32.438.856 [mindspore/ccsrc/frontend/optimizer/ad/bprop_utils.cc:70] GetBprop] Fail to find bprop function for UpdateState. fn: None [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.454.891 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractUMonad(ValueAny)}, g: _zeros_like_u_monad_302 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.468.724 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractUMonad(ValueAny), AbstractUMonad(ValueAny)}, g: hyper_map_303 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.471.314 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractUMonad(ValueAny), AbstractUMonad(ValueAny)}, g: _add_umonad_umonad_304 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.473.917 [mindspore/ccsrc/pipeline/jit/ps/action.cc:282] AbstractAnalyze] function call depth: 0, simulate call depth: 0 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.474.117 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:222] Run] Specialize set top func graph context: {FuncGraph: 1_mindspore_train_dataset_helper__DataWrapper_construct_300 Args: [0]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [1]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [2]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [3]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [4]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [5]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [6]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [7]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [8]: AbstractRefTensor(key: global_step ref_value: AbstractRefTensor(shape: (1), element: AbstractScalar(Type: Int32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [9]: AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [10]: AbstractRefTensor(key: moments.conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [11]: AbstractRefTensor(key: moments.fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [12]: AbstractRefTensor(key: moments.fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [13]: AbstractRefTensor(key: moments.fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [14]: AbstractRefTensor(key: moments.fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [15]: AbstractRefTensor(key: moments.fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [16]: AbstractRefTensor(key: moments.fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [17]: AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [18]: AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), Parent: } [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:37:32.516.894 [mindspore/ccsrc/frontend/parallel/pynative_shard/pynative_shard.cc:334] PynativeShard] Only auto_parallel and semi_auto_parallel support pynative shard [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.522.644 [mindspore/ccsrc/pipeline/jit/ps/action.cc:282] AbstractAnalyze] function call depth: 0, simulate call depth: 0 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:37:32.523.055 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:222] Run] Specialize set top func graph context: {FuncGraph: 251_1_mindspore_train_dataset_helper__DataWrapper_construct_315 Args: [0]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [1]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [2]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [3]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [4]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [5]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [6]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [7]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [8]: AbstractRefTensor(key: global_step ref_value: AbstractRefTensor(shape: (1), element: AbstractScalar(Type: Int32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [9]: AbstractRefTensor(key: moments.conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [10]: AbstractRefTensor(key: moments.conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [11]: AbstractRefTensor(key: moments.fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [12]: AbstractRefTensor(key: moments.fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [13]: AbstractRefTensor(key: moments.fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [14]: AbstractRefTensor(key: moments.fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [15]: AbstractRefTensor(key: moments.fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [16]: AbstractRefTensor(key: moments.fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [17]: AbstractRefTensor(key: momentum ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [18]: AbstractRefTensor(key: learning_rate ref_value: AbstractRefTensor(shape: (), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), Parent: } [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:37:32.529.614 [mindspore/ccsrc/frontend/parallel/pynative_shard/pynative_shard.cc:334] PynativeShard] Only auto_parallel and semi_auto_parallel support pynative shard [INFO] OPTIMIZER(164046,ffff82e63440,python):2024-01-10-11:37:32.534.909 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] OPTIMIZER(164046,ffff82e63440,python):2024-01-10-11:37:32.535.975 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] OPTIMIZER(164046,ffff82e63440,python):2024-01-10-11:37:32.536.795 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:37:32.536.947 [mindspore/ccsrc/frontend/parallel/cache_embedding/cache_embedding.cc:702] AddCacheEmbedding] Parameters are all not cache enable. [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:37:32.537.949 [mindspore/ccsrc/frontend/parallel/pass/assign_add_opt.cc:120] AssignAddOpt] parallel::g_device_manager is not initialized. [INFO] OPTIMIZER(164046,ffff82e63440,python):2024-01-10-11:37:32.538.015 [mindspore/ccsrc/frontend/optimizer/comm_op_reuse_tag.cc:59] AddCommOpReuseTag] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:37:32.538.036 [mindspore/ccsrc/frontend/parallel/pass/overlap_opt_shard_in_pipeline.cc:70] OverlapOptShardInPipeline] parallel::g_device_manager is not initialized. [INFO] OPTIMIZER(164046,ffff82e63440,python):2024-01-10-11:37:32.538.053 [mindspore/ccsrc/frontend/optimizer/grouped_pairwise_exchange_alltoall.cc:673] SetGroupedPairwiseExchangeAllToAll] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:37:32.538.075 [mindspore/ccsrc/frontend/parallel/pass/overlap_gradmatmul_and_gradallreduce.cc:358] OverlapGradMatmulAndGradAllreduce] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:37:32.538.093 [mindspore/ccsrc/frontend/parallel/pass/split_matmul_comm_elementwise_fp.cc:184] SplitMatmulCommElementwiseFp] SplitMatmulCommElementwiseFp is only support under [semi_]auto_parallel, skip it. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.538.123 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end optimize action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.538.144 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start auto_monad_reorder action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.538.481 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end auto_monad_reorder action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.538.514 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start get_jit_bprop_graph action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.538.528 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end get_jit_bprop_graph action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.538.546 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start eliminate_special_op_node action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.539.618 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end eliminate_special_op_node action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.539.662 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start validate action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.539.947 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end validate action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.539.990 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start distribtued_split action. [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:37:32.540.015 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:372] GenerateStrategy] Current parallel mode is data_parallel [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:37:32.540.028 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:384] GenerateStrategy] Generated distributed strategy is 1 [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:37:32.540.390 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:1270] Run] All nodes are on this precoess so there's no need to build and split distributed graph. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.540.422 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end distribtued_split action. [INFO] PROFILER(164046,ffff82e63440,python):2024-01-10-11:37:32.540.489 [mindspore/ccsrc/plugin/device/ascend/hal/profiler/parallel_strategy_profiling.cc:48] IsProfilingParallelStrategyEnabled] Profiling parallel strategy is disabled. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.540.509 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start task_emit action. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.540.878 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.540.902 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.540.914 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1252] SetRunMode] Run graph mode with kernel by kernel by configuration. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:37:32.540.965 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:546] CompileGraphs] Status record: start compile function graph: 381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.541.125 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unify_mindir_pm_0_erase_invalid_micro_depend in 1.46 us [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.199 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.232 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.574 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.595 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.617 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.631 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.648 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.659 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.673 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.695 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.710 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.721 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.736 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.749 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.763 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.773 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.786 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.797 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.810 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.821 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.836 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.847 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.860 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.871 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.896 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.908 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.924 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.935 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.949 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.960 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.975 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.542.988 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.003 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.024 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.040 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.053 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.068 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.081 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.096 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.107 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.122 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.133 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.148 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.159 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.174 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.185 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.200 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.210 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.224 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.235 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.249 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.259 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.274 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.284 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.299 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.309 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.331 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.343 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.361 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.372 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.388 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.401 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.415 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.426 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.441 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.452 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.467 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.478 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.492 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.503 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.518 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.529 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.543 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.554 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.568 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.580 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.595 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.606 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.620 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.637 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.652 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.663 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.676 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.687 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.702 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.713 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.727 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.738 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.753 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.764 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.778 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.789 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.803 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.813 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.826 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.837 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.851 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.862 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.876 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.887 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.902 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.912 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.926 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.942 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.957 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.967 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.981 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.543.991 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.005 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.016 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.033 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.045 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.059 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.070 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.084 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.095 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.109 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.120 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.134 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.145 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.158 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.169 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.182 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.192 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.206 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.217 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.243 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.254 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.268 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.279 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.294 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.304 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.321 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.332 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.346 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.356 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.371 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.381 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.394 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.405 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.419 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.430 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.444 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.454 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.468 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.479 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.493 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.503 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.518 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.533 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.548 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.559 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.573 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.583 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.598 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.608 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.622 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.633 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.646 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.657 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.671 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.681 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.695 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.706 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.720 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.731 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.745 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.756 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.770 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.780 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.793 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.804 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.818 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.833 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.847 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.858 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.872 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.883 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.897 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.908 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.922 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.933 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.948 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.958 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.972 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.982 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.544.997 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.545.007 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.545.021 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.545.031 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.545.044 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.545.055 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.545.068 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.545.078 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.545.091 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.545.102 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:37:32.545.174 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:727] CompileGraph] Compile graph: 381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316, Split segments size: 2 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:37:32.545.216 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:762] CompileGraphFromSegment] Compile normal segment, the first node: @381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316:equiv_CNode_317{[0]: ValueNode Load, [1]: param_fc3.bias, [2]: ValueNode U} [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.545.639 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:421] CompileGraph] Status record: start compile graph. [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.545.678 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:1940] ConstructKernelGraph] Create graph: 1 [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.546.375 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:872] CreateNewValueNode] Data sync for Tensor Tensor(shape=[1], dtype=Int32, value=[1]) [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.546.558 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:872] CreateNewValueNode] Data sync for Tensor Tensor(shape=[], dtype=Float32, value=0.25) [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.547.886 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:2697] ConstructOutput] Output:@381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316:CNode_278{[0]: ValueNode Depend, [1]: CNode_278, [2]: CNode_318} [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.549.817 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:193] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:1 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.550.262 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 83.8 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.550.814 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 503.95 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.551.740 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:154] CommonUnifyMindIR] start common unify mindir opt graph:1 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.552.056 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.552.445 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 384.49 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.552.475 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.552.511 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 31.95 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.552.530 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.552.544 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 13.99 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.552.558 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.553.080 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 506.29 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.554.017 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:79] BackendCommonOptimization] Status record: start common optimization. graph id: 1 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.554.340 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: add_dynamic_shape_attr [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.555.084 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_0_add_dynamic_shape_attr in 736.97 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.555.118 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.555.166 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_1_convert_dynamic_broadcast_to in 43.57 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.555.720 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_2_convert_const_input_to_attr in 522.68 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.556.196 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_3_custom_op_const_input_to_attr in 440 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.556.601 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 370.63 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.557.943 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 1300.15 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.557.999 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 15.9 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.558.365 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_7_flatten_concat_fission in 335.01 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.558.715 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 314.46 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.559.058 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_9_broadcast_to_fusion in 308.91 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.559.536 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 441.51 us [WARNING] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.560.418 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:92] BackendCommonOptimization] [PROF]backend_common_optimization costs 6401 usec. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.560.452 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:99] BackendCommonOptimization] Status record: end common optimization. graph id: 1 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.561.191 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_0_renorm_split in 341.2 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.561.224 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.562.113 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_1_reduce_axis_update in 875.95 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.562.159 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim HistogramFixedWidth not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.562.179 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_2_histogram_fixed_width_fusion in 20.93 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.562.194 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim ClipByNorm not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.562.208 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_3_clip_by_norm_fission in 13.15 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.562.222 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.562.235 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_4_tensor_arry_add_flow_cond1 in 12.86 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.562.247 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayGather not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.562.260 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_5_tensor_arry_add_flow_cond2 in 11.25 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.562.273 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.562.285 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_6_ge_tensor_arry_cast_index in 11.56 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.562.298 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArray not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.562.311 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_7_tensor_array_prepare in 11.89 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.562.323 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.562.360 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_8_space_to_batch_nd_attr_update in 33.15 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.562.376 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.562.403 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_9_batch_to_space_nd_attr_update in 24.37 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.562.416 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: avg_pool_grad_unify_mindir [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.562.450 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_10_avg_pool_grad_unify_mindir in 31.25 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.562.815 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_11_adam_weight_decay_unify_mindir in 339.95 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.563.166 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_12_cdist_fission in 319.29 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.563.523 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_13_cdist_grad_fission in 316.65 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.563.969 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_14_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 413.41 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.565.129 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_15_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 1114.18 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.565.552 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_16_sparse_softmax_cross_entropy_with_logits_unify_mindir in 374.63 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.565.952 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_17_dropout_unify_mindir1 in 359.72 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.565.983 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.348 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_18_dropoutgrad_unify_mindir in 354.43 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.375 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.393 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_19_neighbor_exchange_unify_mindir in 16.72 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.406 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.420 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_20_neighbor_exchange_v2_unify_mindir in 12.04 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.432 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.445 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_21_neighbor_exchange_v2_grad_unify_mindir in 11.13 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.457 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.470 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_22_all_to_all_unify_mindir in 11.17 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.485 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim QuantDTypeCast not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.497 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_23_quant_dtype_cst_adjust in 14.03 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.512 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim FSEDecode not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.524 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_24_fse_decode_adjust in 13.37 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.537 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.560 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_25_bn_split in 21.73 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.573 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.631 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_26_bn_grad_unify_mindir in 53.37 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.649 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.663 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_27_bn_grad_split in 13.88 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.677 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.690 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_28_batchnorm_to_bninfer in 12.06 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.703 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.715 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_29_batchnormgrad_to_bninfergrad in 11.37 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.728 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.566.740 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_30_batch_norm_grad_infer_fission in 11.64 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.567.598 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_31_ascend_mindir_op_adapter in 825.86 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.567.635 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_32_DropoutGenMask in 0.93 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.568.023 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_33_lamb_fission_ge in 361.32 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.568.877 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_34_print_insert_placeholder_for_tensor_name in 817.19 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.569.287 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_35_getnext_for_ge in 371.79 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.569.661 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_36_sync_bn_split in 338.02 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.570.046 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_37_sync_bn_grad_split in 349.78 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.570.412 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_38_adaptive_max_pool2d_ge_fusion in 331.83 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.570.778 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_39_avg_pool_grad_for_ge in 330.82 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.570.820 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:260] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.571.698 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_40_FlashAttentionFusionV1 in 871.86 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.571.732 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:374] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.572.589 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_41_FlashAttentionFusionV1 in 850.9 us [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.574.645 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:37:32.574.681 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.574.698 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:132] GetRunMode] RunMode::kKernelMode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.575.913 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 1166.67 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.577.678 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_0_seed_adapter in 1103.62 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.577.742 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: env_op_attr_update [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.578.507 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_1_env_op_attr_update in 749.77 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.578.916 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_2_process call inline in 362.07 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.579.273 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_3_expander_fallback in 313.7 us [WARNING] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.579.891 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:139] GEBackendOptimizeACL] [PROF]ascend_backend_optimize_acl costs 3334 usec. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.579.952 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] GetNext is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.580.511 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2D is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.580.876 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPool is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.581.043 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2D is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.581.181 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPool is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.581.387 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.581.850 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.582.042 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.582.234 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] SoftmaxCrossEntropyWithLogits is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.582.785 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.582.908 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.583.020 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.583.276 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.583.580 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.583.674 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.583.761 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.583.958 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.584.160 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.584.243 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.584.349 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.584.541 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.584.746 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPoolGrad is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.585.005 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AssignAdd is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.585.141 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2DBackpropInput is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.585.326 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2DBackpropFilter is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.585.525 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.585.734 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.585.935 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPoolGrad is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.586.095 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2DBackpropFilter is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.586.213 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] AllReduce is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.586.371 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.586.474 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.586.571 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.586.668 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.586.762 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.586.856 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.586.995 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] ApplyMomentum is not defined in opdef. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.587.536 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 150.83 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.588.416 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 834.44 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.589.246 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 789.85 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.590.739 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 1442.94 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.592.117 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_4_insert_type_transform_op in 1324.76 us [WARNING] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.592.738 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:179] GEBackendOptimizeACLAfterKernelSelect] [PROF]ascend_backend_optimize_acl_after_kernel_select costs 5393 usec. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.592.846 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_0_DropoutGenMask in 0.71 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.593.189 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_1_cse in 300.84 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.594.395 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_redundant_op in 1150.14 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.594.598 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_3_eliminate_maketuple_getitem in 159.07 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.594.630 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_4_MergeTransData in 0.89 us [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.595.603 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive GetNext [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:32.595.870 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:467] ConvertAny] Value: ValueTuple [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.595.994 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive GetNext [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.596.025 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.596.134 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.596.160 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive OneHot [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.596.319 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive OneHot [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.596.346 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2D [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:32.596.412 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:32.596.455 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:32.596.482 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.596.556 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2D [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.596.579 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.596.646 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.596.668 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPool [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.596.773 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPool [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.596.797 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2D [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:32.596.848 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:32.596.892 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:32.596.917 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.596.982 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2D [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.597.003 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.597.067 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.597.088 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPool [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.597.165 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPool [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.597.186 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Flatten [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.597.262 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Flatten [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.597.283 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.597.395 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.597.419 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.597.574 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.597.601 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.597.666 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.597.739 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.597.827 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.597.849 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.597.939 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.597.961 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.598.032 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.598.053 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.598.134 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.598.156 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.598.245 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.598.268 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.598.341 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.598.362 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive SoftmaxCrossEntropyWithLogits [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.598.462 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive SoftmaxCrossEntropyWithLogits [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.598.483 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.598.559 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.598.580 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.598.648 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.598.668 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReduceMean [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.598.770 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReduceMean [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.598.794 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.598.888 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.598.911 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.598.996 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.599.029 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.599.181 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.599.286 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.599.311 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.599.364 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.599.438 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.599.460 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReluGrad [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.599.532 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReluGrad [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.599.553 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.599.629 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.599.650 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.599.721 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.599.742 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.599.795 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.599.864 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.599.885 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.599.960 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.599.983 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.600.035 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.600.101 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.600.132 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReluGrad [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.600.203 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReluGrad [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.600.223 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.600.295 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.600.316 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.600.397 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.600.418 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.600.472 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.600.539 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.600.560 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.600.633 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAddGrad [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.600.655 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.600.706 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.600.775 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.600.796 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.600.880 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.600.901 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPoolGrad [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.601.021 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPoolGrad [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.601.044 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReluGrad [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.601.123 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReluGrad [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.601.143 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive AssignAdd [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.601.215 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive AssignAdd [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.601.235 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2DBackpropInput [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:32.601.310 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:32.601.341 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.601.417 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2DBackpropInput [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.601.440 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2DBackpropFilter [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:32.601.500 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:32.601.536 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:32.601.561 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.601.634 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2DBackpropFilter [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.601.656 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.601.740 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.601.816 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.601.837 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.601.963 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.601.988 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPoolGrad [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.602.087 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPoolGrad [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.602.110 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReluGrad [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.602.180 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReluGrad [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.602.218 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2DBackpropFilter [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:32.602.276 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:32.602.312 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:37:32.602.336 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.602.413 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2DBackpropFilter [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.602.436 [mindspore/ccsrc/plugin/device/ascend/kernel/hccl/hccl_kernel_build.cc:31] HcclOpBuild] Build hccl op [AllReduce] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.602.494 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.602.568 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.602.592 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.602.690 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.602.714 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.602.804 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.602.828 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.602.918 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.602.940 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.603.028 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.603.051 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.603.137 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.603.158 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.603.241 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.603.272 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Mul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.603.338 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Mul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.603.357 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ApplyMomentum [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.603.442 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ApplyMomentum [WARNING] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.603.460 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:262] CreateKernel] [PROF]create_kernel costs 7898 usec. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.603.693 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/Reshape-op0, index: 0 to input Default/GetNext-op1, index: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.603.728 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1, index: 0 to input Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2, index: 0 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.603.753 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/Reshape-op1, index: 0 to input Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1, index: 0 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.603.775 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/Reshape-op2, index: 0 to input Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/Mul-op0, index: 0 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.603.796 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/gradFlatten-expand/Reshape-op1, index: 0 to input Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op6, index: 0 [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.603.811 [mindspore/ccsrc/backend/common/session/session_basic.cc:1140] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 1 start [INFO] DEBUG(164046,ffff82e63440,python):2024-01-10-11:37:32.603.823 [mindspore/ccsrc/debug/summary/summary.cc:33] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 1 start [INFO] DEBUG(164046,ffff82e63440,python):2024-01-10-11:37:32.603.836 [mindspore/ccsrc/debug/summary/summary.cc:54] RecurseSetSummaryNodesForAllGraphs] The total summary nodes is: 0 for graph: 1 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.604.425 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8's input.Send node Default/StreamSend-op0 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op7, recv node Default/StreamRecv-op0 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.604.505 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8's output.Send node Default/StreamSend-op1 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8, recv node Default/StreamRecv-op1 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op8 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.604.559 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9's input.Send node Default/StreamSend-op2 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op3, recv node Default/StreamRecv-op2 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.604.603 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9's output.Send node Default/StreamSend-op3 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9, recv node Default/StreamRecv-op3 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op9 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.604.655 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10's input.Send node Default/StreamSend-op4 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op8, recv node Default/StreamRecv-op4 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.604.699 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10's output.Send node Default/StreamSend-op5 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10, recv node Default/StreamRecv-op5 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op10 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.604.750 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11's input.Send node Default/StreamSend-op6 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op4, recv node Default/StreamRecv-op6 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.604.796 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11's output.Send node Default/StreamSend-op7 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11, recv node Default/StreamRecv-op7 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op11 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.604.848 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12's input.Send node Default/StreamSend-op8 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op9, recv node Default/StreamRecv-op8 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.604.901 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12's output.Send node Default/StreamSend-op9 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12, recv node Default/StreamRecv-op9 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op12 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.604.949 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13's input.Send node Default/StreamSend-op10 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op5, recv node Default/StreamRecv-op10 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.604.992 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13's output.Send node Default/StreamSend-op11 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13, recv node Default/StreamRecv-op11 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op13 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.605.052 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14's input.Send node Default/StreamSend-op12 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/gradConv2D-expand/Conv2DBackpropFilter-op1, recv node Default/StreamRecv-op12 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.605.097 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14's output.Send node Default/StreamSend-op13 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14, recv node Default/StreamRecv-op13 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op14 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.605.150 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:270] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15's input.Send node Default/StreamSend-op14 after Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv1-Conv2d/gradConv2D-expand/Conv2DBackpropFilter-op1, recv node Default/StreamRecv-op14 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.605.195 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/acl_stream_assign.cc:264] InsertEvents] Generate send/recv for parallel op Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15's output.Send node Default/StreamSend-op15 after Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15, recv node Default/StreamRecv-op15 before Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op15 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.318 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.394 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:0 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.418 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.437 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:0 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.456 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.472 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:1 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.489 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.504 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:2 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.521 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.535 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:2 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.552 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.565 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:3 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.581 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.595 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:1 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.615 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.628 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:4 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.644 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.657 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:4 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.673 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.696 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:5 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.719 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.734 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:3 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.751 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.785 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:6 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.802 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.816 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:6 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.833 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.846 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:7 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.863 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.877 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:5 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.896 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.909 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:8 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.926 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.939 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:8 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.955 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.969 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:9 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.985 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.605.998 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:7 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.015 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.029 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:10 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.046 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.059 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:10 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.076 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.089 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:11 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.105 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.118 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:9 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.139 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.163 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:12 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.180 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.193 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:12 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.209 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.223 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:13 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.239 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.252 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:11 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.270 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.284 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:14 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.299 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.312 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:14 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.329 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamsend] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.342 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/send.cc:41] Init] send op event id:15 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.357 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.370 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:13 [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.391 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/rt_kernel_build.cc:31] RtOpBuild] Op Name(tolower)[streamrecv] [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:37:32.606.405 [mindspore/ccsrc/plugin/device/ascend/kernel/rts/recv.cc:42] Init] recv op event_id_:15 [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.606.423 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:891] PrintGraphExecuteOrder] Graph 1 execution order: [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.606.462 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[0], node name[Default/GetNext-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:outputs{[0]: ValueNode GetNext}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.606.514 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[1], node name[Default/Reshape-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_Reshape, [1]: CNode_278, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[32])}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.606.576 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[2], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/OneHot-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_OneHot, [1]: 319, [2]: ValueNode Tensor(shape=[], dtype=Int64, value=10), [3]: ValueNode Tensor(shape=[], dtype=Float32, value=1), [4]: ValueNode Tensor(shape=[], dtype=Float32, value=0), [5]: ValueNode -1}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.606.638 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[3], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv1-Conv2d/Conv2D-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode Conv2D, [1]: CNode_278, [2]: equiv_CNode_320}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.606.683 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[4], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op4], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_ReLU, [1]: equiv_CNode_278}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.606.729 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[5], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op3], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode MaxPool, [1]: equiv_CNode_278}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.606.781 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[6], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/Conv2D-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode Conv2D, [1]: equiv_CNode_278, [2]: equiv_CNode_321}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.606.823 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[7], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op5], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_ReLU, [1]: equiv_CNode_278}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.606.866 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[8], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode MaxPool, [1]: equiv_CNode_278}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.606.918 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[9], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_Flatten, [1]: equiv_CNode_278}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.606.974 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[10], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/MatMul-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode MatMul, [1]: equiv_CNode_278, [2]: equiv_CNode_322}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.028 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[11], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/BiasAdd-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_BiasAdd, [1]: equiv_CNode_278, [2]: equiv_CNode_323, [3]: ValueNode 0}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.084 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[12], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op6], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_ReLU, [1]: equiv_CNode_278}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.135 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[13], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/MatMul-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode MatMul, [1]: equiv_CNode_278, [2]: equiv_CNode_324}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.188 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[14], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/BiasAdd-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_BiasAdd, [1]: equiv_CNode_278, [2]: equiv_CNode_325, [3]: ValueNode 0}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.231 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[15], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op7], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_ReLU, [1]: equiv_CNode_278}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.279 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[16], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/MatMul-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode MatMul, [1]: equiv_CNode_278, [2]: equiv_CNode_326}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.327 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[17], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:equiv_CNode_278{[0]: ValueNode PrimFunc_BiasAdd, [1]: equiv_CNode_278, [2]: equiv_CNode_317, [3]: ValueNode 0}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.373 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[18], node name[Default/Reshape-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_Reshape, [1]: equiv_CNode_278, [2]: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10])}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.409 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[19], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/SoftmaxCrossEntropyWithLogits-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode SoftmaxCrossEntropyWithLogits, [1]: 319, [2]: 319}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.446 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[20], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/Mul-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_Mul, [1]: 319, [2]: ValueNode Tensor(shape=[32, 1], dtype=Float32, value=[...])}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.491 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[21], node name[Default/Reshape-op2], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_Reshape, [1]: 319, [2]: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10])}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.531 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[22], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/ReduceMean-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:319{[0]: ValueNode PrimFunc_ReduceMean, [1]: 319, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), [3]: ValueNode false}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.568 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[23], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op10], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 319, [2]: equiv_CNode_326}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.606 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[24], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op7], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 319, [2]: equiv_CNode_278}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.632 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[25], node name[Default/StreamSend-op0], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_328{[0]: ValueNode StreamSend}], event id[0] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.655 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[26], node name[Default/StreamRecv-op0], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_329{[0]: ValueNode StreamRecv}], event id[0] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.694 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[27], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 327}], group[hccl_world_group] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.718 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[28], node name[Default/StreamSend-op1], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_330{[0]: ValueNode StreamSend}], event id[1] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.751 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[29], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op3], logic id[4294967295], stream id[0], node info[@kernel_graph_1:331{[0]: ValueNode PrimFunc_BiasAddGrad, [1]: 319, [2]: ValueNode 0}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.776 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[30], node name[Default/StreamSend-op2], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_332{[0]: ValueNode StreamSend}], event id[2] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.806 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[31], node name[Default/StreamRecv-op2], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_333{[0]: ValueNode StreamRecv}], event id[2] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.845 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[32], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 331}], group[hccl_world_group] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.871 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[33], node name[Default/StreamSend-op3], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_334{[0]: ValueNode StreamSend}], event id[3] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.897 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[34], node name[Default/StreamRecv-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_335{[0]: ValueNode StreamRecv}], event id[1] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.607.954 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[35], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op8], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.003 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[36], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/gradReLU-expand/ReluGrad-op4], logic id[4294967295], stream id[0], node info[@kernel_graph_1:336{[0]: ValueNode PrimFunc_ReluGrad, [1]: 327, [2]: equiv_CNode_278}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.040 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[37], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op11], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 336, [2]: equiv_CNode_324}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.081 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[38], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op8], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 336, [2]: equiv_CNode_278}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.105 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[39], node name[Default/StreamSend-op4], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_337{[0]: ValueNode StreamSend}], event id[4] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.131 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[40], node name[Default/StreamRecv-op4], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_338{[0]: ValueNode StreamRecv}], event id[4] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.171 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[41], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 327}], group[hccl_world_group] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.205 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[42], node name[Default/StreamSend-op5], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_339{[0]: ValueNode StreamSend}], event id[5] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.230 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[43], node name[Default/StreamRecv-op3], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_340{[0]: ValueNode StreamRecv}], event id[3] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.283 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[44], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op9], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.317 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[45], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op4], logic id[4294967295], stream id[0], node info[@kernel_graph_1:331{[0]: ValueNode PrimFunc_BiasAddGrad, [1]: 336, [2]: ValueNode 0}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.340 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[46], node name[Default/StreamSend-op6], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_341{[0]: ValueNode StreamSend}], event id[6] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.362 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[47], node name[Default/StreamRecv-op6], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_342{[0]: ValueNode StreamRecv}], event id[6] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.399 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[48], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 331}], group[hccl_world_group] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.421 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[49], node name[Default/StreamSend-op7], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_343{[0]: ValueNode StreamSend}], event id[7] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.444 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[50], node name[Default/StreamRecv-op5], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_344{[0]: ValueNode StreamRecv}], event id[5] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.495 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[51], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op10], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.542 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[52], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/gradReLU-expand/ReluGrad-op5], logic id[4294967295], stream id[0], node info[@kernel_graph_1:336{[0]: ValueNode PrimFunc_ReluGrad, [1]: 327, [2]: equiv_CNode_278}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.586 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[53], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op6], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 336, [2]: equiv_CNode_322}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.638 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[54], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op9], logic id[4294967295], stream id[0], node info[@kernel_graph_1:327{[0]: ValueNode MatMul, [1]: 336, [2]: equiv_CNode_278}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.663 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[55], node name[Default/StreamSend-op8], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_345{[0]: ValueNode StreamSend}], event id[8] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.686 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[56], node name[Default/StreamRecv-op8], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_346{[0]: ValueNode StreamRecv}], event id[8] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.723 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[57], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 327}], group[hccl_world_group] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.748 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[58], node name[Default/StreamSend-op9], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_347{[0]: ValueNode StreamSend}], event id[9] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.774 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[59], node name[Default/StreamRecv-op7], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_348{[0]: ValueNode StreamRecv}], event id[7] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.823 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[60], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op11], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.858 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[61], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradBiasAdd-expand/BiasAddGrad-op5], logic id[4294967295], stream id[0], node info[@kernel_graph_1:331{[0]: ValueNode PrimFunc_BiasAddGrad, [1]: 336, [2]: ValueNode 0}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.883 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[62], node name[Default/StreamSend-op10], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_349{[0]: ValueNode StreamSend}], event id[10] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.907 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[63], node name[Default/StreamRecv-op10], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_350{[0]: ValueNode StreamRecv}], event id[10] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.954 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[64], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 331}], group[hccl_world_group] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.608.978 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[65], node name[Default/StreamSend-op11], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_351{[0]: ValueNode StreamSend}], event id[11] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.000 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[66], node name[Default/StreamRecv-op9], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_352{[0]: ValueNode StreamRecv}], event id[9] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.049 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[67], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op12], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.084 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[68], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/gradFlatten-expand/Reshape-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:353{[0]: ValueNode PrimFunc_Reshape, [1]: 327, [2]: ValueNode (32, 16, 5, 5)}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.141 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[69], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/gradMaxPool-expand/MaxPoolGrad-op2], logic id[4294967295], stream id[0], node info[@kernel_graph_1:354{[0]: ValueNode MaxPoolGrad, [1]: equiv_CNode_278, [2]: equiv_CNode_278, [3]: 353}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.186 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[70], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/gradReLU-expand/ReluGrad-op6], logic id[4294967295], stream id[0], node info[@kernel_graph_1:336{[0]: ValueNode PrimFunc_ReluGrad, [1]: 354, [2]: equiv_CNode_278}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.234 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[71], node name[Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AssignAdd, [1]: param_global_step, [2]: ValueNode Tensor(shape=[1], dtype=Int32, value=[1]), [3]: CNode_355}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.277 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[72], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/gradConv2D-expand/Conv2DBackpropInput-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:356{[0]: ValueNode Conv2DBackpropInput, [1]: 336, [2]: equiv_CNode_321, [3]: ValueNode (32, 6, 14, 14)}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.333 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[73], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/gradConv2D-expand/Conv2DBackpropFilter-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:356{[0]: ValueNode Conv2DBackpropFilter, [1]: 336, [2]: equiv_CNode_278, [3]: ValueNode (16, 6, 5, 5)}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.361 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[74], node name[Default/StreamSend-op12], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_357{[0]: ValueNode StreamSend}], event id[12] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.388 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[75], node name[Default/StreamRecv-op12], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_358{[0]: ValueNode StreamRecv}], event id[12] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.425 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[76], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 356}], group[hccl_world_group] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.448 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[77], node name[Default/StreamSend-op13], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_359{[0]: ValueNode StreamSend}], event id[13] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.470 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[78], node name[Default/StreamRecv-op11], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_360{[0]: ValueNode StreamRecv}], event id[11] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.520 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[79], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op13], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.589 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[80], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc3.bias, [2]: param_moments.fc3.bias, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_361}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.646 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[81], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/gradMaxPool-expand/MaxPoolGrad-op3], logic id[4294967295], stream id[0], node info[@kernel_graph_1:354{[0]: ValueNode MaxPoolGrad, [1]: equiv_CNode_278, [2]: equiv_CNode_278, [3]: 356}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.730 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[82], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/gradReLU-expand/ReluGrad-op7], logic id[4294967295], stream id[0], node info[@kernel_graph_1:336{[0]: ValueNode PrimFunc_ReluGrad, [1]: 354, [2]: equiv_CNode_278}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.780 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[83], node name[Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv1-Conv2d/gradConv2D-expand/Conv2DBackpropFilter-op1], logic id[4294967295], stream id[0], node info[@kernel_graph_1:362{[0]: ValueNode Conv2DBackpropFilter, [1]: 336, [2]: CNode_278, [3]: ValueNode (6, 1, 5, 5)}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.807 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[84], node name[Default/StreamSend-op14], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_363{[0]: ValueNode StreamSend}], event id[14] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.830 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[85], node name[Default/StreamRecv-op14], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_364{[0]: ValueNode StreamRecv}], event id[14] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.867 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[86], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_278{[0]: ValueNode AllReduce, [1]: 362}], group[hccl_world_group] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.893 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[87], node name[Default/StreamSend-op15], logic id[4294967295], stream id[1], node info[@kernel_graph_1:CNode_365{[0]: ValueNode StreamSend}], event id[15] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.916 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[88], node name[Default/StreamRecv-op13], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_366{[0]: ValueNode StreamRecv}], event id[13] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.609.968 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[89], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op14], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.610.040 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[90], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc3.weight, [2]: param_moments.fc3.weight, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_367}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.610.109 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[91], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc2.bias, [2]: param_moments.fc2.bias, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_368}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.610.179 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[92], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc2.weight, [2]: param_moments.fc2.weight, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_369}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.610.257 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[93], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc1.bias, [2]: param_moments.fc1.bias, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_370}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.610.328 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[94], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_fc1.weight, [2]: param_moments.fc1.weight, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_371}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.610.394 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[95], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_conv2.weight, [2]: param_moments.conv2.weight, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_372}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.610.419 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[96], node name[Default/StreamRecv-op15], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_373{[0]: ValueNode StreamRecv}], event id[15] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.610.470 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[97], node name[Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/Mul-op15], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode PrimFunc_Mul, [1]: CNode_278, [2]: ValueNode Tensor(shape=[], dtype=Float32, value=0.25)}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.610.538 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[98], node name[Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15], logic id[4294967295], stream id[0], node info[@kernel_graph_1:CNode_278{[0]: ValueNode ApplyMomentum, [1]: param_conv1.weight, [2]: param_moments.conv1.weight, [3]: param_learning_rate, [4]: CNode_278, [5]: param_momentum, [6]: CNode_374}] [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.611.006 [mindspore/ccsrc/backend/common/somas/somas.cc:143] Assign] Start Somas Assign for graph 1 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.611.412 [mindspore/ccsrc/backend/common/somas/somas.cc:151] Assign] Somas Configure success, configuration info: Device Name: Ascend Run by execution order: 1 Enable debug log: 0 Debug log path: . [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.611.432 [mindspore/ccsrc/backend/common/somas/somas.cc:154] Assign] Start Initialize SOMAS Model [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.612.713 [mindspore/ccsrc/backend/common/somas/somas.cc:1065] GraphOutputProcess] Set 1 graph output tensors' aligned size to 0. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.613.325 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 1 output 2 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.613.362 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 9 output 10 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.613.388 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 18 output 19 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.613.415 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 22 output 23 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.613.459 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 40 output 47 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.613.494 [mindspore/ccsrc/backend/common/somas/somas.cc:1135] RefNodeProcess] Special Tensor total size: RefNode: input 107008 output 351828 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.613.551 [mindspore/ccsrc/backend/common/somas/somas.cc:555] InitSomasModel] Created 2 streams (0 groups), 99 nodes, 69 tensors, 5 union tensors lists, and 0 contiguous tensors lists [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.614.353 [mindspore/ccsrc/backend/common/somas/somas.cc:157] Assign] End Initialize SOMAS Model [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.614.377 [mindspore/ccsrc/backend/common/somas/somas.cc:176] Assign] Start Computing Conflict Matrix [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.614.389 [mindspore/ccsrc/backend/common/somas/somas.cc:1286] ComputeBasicMatrix] Start Conflict Computing (Bitset Model) [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.614.410 [mindspore/ccsrc/backend/common/somas/somas.cc:1291] ComputeBasicMatrix] Start Bitset [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.614.446 [mindspore/ccsrc/backend/common/somas/somas.cc:1299] ComputeBasicMatrix] Start Path Computing [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.614.471 [mindspore/ccsrc/backend/common/somas/somas.cc:1307] ComputeBasicMatrix] End Path Computing [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.614.481 [mindspore/ccsrc/backend/common/somas/somas.cc:1309] ComputeBasicMatrix] Start Tensor Relation Computing [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.614.548 [mindspore/ccsrc/backend/common/somas/somas.cc:1462] ComputeMultiTensorConflicts] Start Computing Conflicts Pairs, tensors list size is 60 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.614.623 [mindspore/ccsrc/backend/common/somas/somas.cc:1469] ComputeMultiTensorConflicts] End Computing Conflicts Pairs (time taken 0ms) [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.614.634 [mindspore/ccsrc/backend/common/somas/somas.cc:1367] ComputeBasicMatrix] End Basic Conflict Computing (Bitset Model)(time taken 0ms) [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.614.663 [mindspore/ccsrc/backend/common/somas/somas.cc:178] Assign] End Computing Conflict Matrix [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.614.675 [mindspore/ccsrc/backend/common/somas/somas.cc:1533] Solve] Somas Assign start... [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.614.705 [mindspore/ccsrc/backend/common/somas/somas.cc:1555] Solve] Start Solving [INFO] PRE_ACT(164046,fffea17fa0f0,python):2024-01-10-11:37:32.614.981 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 55 tensors [INFO] PRE_ACT(164046,fffe87fff0f0,python):2024-01-10-11:37:32.614.995 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 55 tensors [INFO] PRE_ACT(164046,fffe877fe0f0,python):2024-01-10-11:37:32.615.007 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 55 tensors [INFO] PRE_ACT(164046,fffea0ff90f0,python):2024-01-10-11:37:32.615.015 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 55 tensors [INFO] PRE_ACT(164046,fffea17fa0f0,python):2024-01-10-11:37:32.615.156 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 1/4 2196992 Bytes (0.00204611 GB) Shared Objects size(>), index(<) bestfit [INFO] PRE_ACT(164046,fffe877fe0f0,python):2024-01-10-11:37:32.615.188 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 4/4 2196992 Bytes (0.00204611 GB) Single Object size(>), index(<) smallest [INFO] PRE_ACT(164046,fffe87fff0f0,python):2024-01-10-11:37:32.615.190 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 2/4 2196992 Bytes (0.00204611 GB) Shared Objects size(>), index(<) smallest [INFO] PRE_ACT(164046,fffea0ff90f0,python):2024-01-10-11:37:32.615.198 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 3/4 2196992 Bytes (0.00204611 GB) Single Object size(>), index(<) bestfit [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.252 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:176] Solving] SOMAS SOLVER RESUME: [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.268 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:177] Solving] Best Solution:[1/4] [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.286 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:178] Solving] Best result:2196992 Bytes 0.00204611 GB (0.00204611 GB + 0 GB from lifelong tensors) [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.300 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:181] Solving] Best timing:0 ms [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.311 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:182] Solving] Best algorithm: Shared Objects [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.321 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:183] Solving] Best sorting strategy: size(>), index(<) [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.331 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:184] Solving] Best offset strategy: bestfit [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.344 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:185] Solving] Time elapsed: 0 ms [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.357 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:186] Solving] Spread:0 %% [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.419 [mindspore/ccsrc/backend/common/somas/somas.cc:1564] Solve] End Solving [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.491 [mindspore/ccsrc/backend/common/somas/somas.cc:2096] GenGraphStatisticInfo] Lower Bound: 2186752 (0.00203657 GB), Upper Bound: 4660224 (0.00434017 GB) [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.504 [mindspore/ccsrc/backend/common/somas/somas.cc:2099] GenGraphStatisticInfo] Total Dynamic Size (Upper Bound): 4660224 Theoretical Optimal Size (Lower Bound): 2186752 Total Workspace Size: 0 Total Communication Input Tensor Size: 248832 Total Communication Output Tensor Size: 0 Total LifeLong All Tensor Size: 0 Total LifeLong Start Tensor Size: 0 Total LifeLong End Tensor Size: 512 Reused Size(Allocate Size): 0 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.516 [mindspore/ccsrc/backend/common/somas/somas.cc:1583] Solve] Somas Assign end. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.648 [mindspore/ccsrc/backend/common/somas/somas.cc:380] UpdateSomasResultToGraph] Merged Block size: 12 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.661 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 0, offset: 1205248, size: 602624 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.674 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 1, offset: 602624, size: 602624 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.686 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 2, offset: 0, size: 602624 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.698 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 3, offset: 1807872, size: 192512 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.719 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 4, offset: 2000384, size: 131584 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.729 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 5, offset: 2131968, size: 40448 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.739 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 6, offset: 2182144, size: 9728 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.749 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 7, offset: 2172416, size: 9728 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.759 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 8, offset: 2191872, size: 3584 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.769 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 9, offset: 2196480, size: 512 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:37:32.615.780 [mindspore/ccsrc/backend/common/somas/somas.cc:189] Assign] Somas Allocate end. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.615.792 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:310] DoSomas] Somas allocate success for graph 1 somas size: 2196992 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.615.984 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:801] CreateDeviceAddress] Status record: start create device address. graph id: 1 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.616.897 [mindspore/ccsrc/runtime/device/device_address_utils.cc:454] CreateValueNodeDeviceAddress] No device address for value node:Default/data-17, debug name:ValueNode U [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.045 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is global_step, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.079 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is global_step, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.103 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2, index is 0; cur kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.131 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2, index is 0; cur kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.149 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc1.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.166 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc1.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.193 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1, index is 0; cur kernel is Default/Reshape-op1, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.216 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1, index is 0; cur kernel is Default/Reshape-op1, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.234 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/GetNext-op1, index is 1; cur kernel is Default/Reshape-op0, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.260 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/GetNext-op1, index is 1; cur kernel is Default/Reshape-op0, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.281 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc2.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.297 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc2.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.312 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc3.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.327 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc3.bias, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.341 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op6, index is 0; cur kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/gradFlatten-expand/Reshape-op1, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.363 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op6, index is 0; cur kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/gradFlatten-expand/Reshape-op1, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.380 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is conv1.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.396 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is conv1.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.411 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc2.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.435 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc2.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.449 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc3.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.464 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc3.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.478 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is conv2.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.494 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is conv2.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.507 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is fc1.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.523 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is fc1.weight, index is 0; cur kernel is Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.536 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/Mul-op0, index is 0; cur kernel is Default/Reshape-op2, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:37:32.619.557 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/Mul-op0, index is 0; cur kernel is Default/Reshape-op2, index is 0 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.619.575 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:808] CreateDeviceAddress] Status record: end create device address. graph id: 1 [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.619.752 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1129] CacheGraphOutputToFrontNodeWithIndex] Get graph backend output nodes. [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.619.786 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1137] CacheGraphOutputToFrontNodeWithIndex] Get graph front output nodes. [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:37:32.619.810 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1155] CacheGraphOutputToFrontNodeWithIndex] Backend output: Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/ReduceMean-op0 with index: 0 map to front node: Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits-op0 with index: 0 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.619.924 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:507] CompileGraph] Status record: end compile graph. graph id: 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:37:32.620.376 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:805] CompileGraphFromSegment] Compile cut segment, the cut node: @381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316:CNode_375{[0]: ValueNode Return, [1]: CNode_278} [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.620.545 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:521] Transform] Graph(kernel_graph_1) transforms actor begin, strategy:pipeline_with_execution_order [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.620.620 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2619] PersistDeviceTensorForValueNode] The device address is not exist: ValueNode_376(U) [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.620.786 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1101] BuildDataSourceActor] Create queue data source actor: kernel_graph_1_DeviceDSActor_1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.621.661 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1232] BuildLoopCountActor] Create loop count actor: kernel_graph_1_LoopCountActor [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.621.698 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1265] BuildOutputActor] Create output actor: kernel_graph_1_OutputActor [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.621.720 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1285] BuildDataPrepareActor] Create data prepare actor: kernel_graph_1_DataPrepareActor [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.621.844 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:866] CacheGraphOutputToActor] Cache graph 1 output node:Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/ReduceMean-op0 debug string:@kernel_graph_1:319{[0]: ValueNode PrimFunc_ReduceMean, [1]: 319, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), [3]: ValueNode false} with index:0 to actor:Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/ReduceMean-op0, from front node:Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits-op0 debug string:@381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316:equiv_CNode_278{[0]: ValueNode SparseSoftmaxCrossEntropyWithLogits, [1]: equiv_CNode_278, [2]: CNode_278} with index:0 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.621.864 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:809] AddSomasInfoForGraphOutput] The graph 1 output node:Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/gradSparseSoftmaxCrossEntropyWithLogits-expand/ReduceMean-op0 with index: 0 somas enable or not: 1, somas offset: 2195456, aligned size: 512 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.621.886 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1518] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_1 start. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.622.024 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1520] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_1 end. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.030 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1 add input node monad device tensor store:global_step [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.058 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1 add input node monad device tensor store:Default/network-TrainOneStepCell/optimizer-Momentum/data-0 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.210 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/AssignAdd-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.237 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.258 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 add input node monad device tensor store:fc3.bias [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.273 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 add input node monad device tensor store:moments.fc3.bias [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.285 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.297 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.470 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op8, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.496 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/MatMul-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.518 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op10, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.546 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 add input node monad device tensor store:fc3.weight [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.559 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 add input node monad device tensor store:moments.fc3.weight [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.570 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.581 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.632 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op9, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.655 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/BiasAdd-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.672 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 add input node monad device tensor store:fc2.bias [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.686 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 add input node monad device tensor store:moments.fc2.bias [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.696 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.707 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.756 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op10, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.786 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/MatMul-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.808 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op11, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.825 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 add input node monad device tensor store:fc2.weight [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.836 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 add input node monad device tensor store:moments.fc2.weight [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.847 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.857 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.907 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op11, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.929 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/BiasAdd-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.946 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 add input node monad device tensor store:fc1.bias [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.960 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 add input node monad device tensor store:moments.fc1.bias [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.971 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.623.989 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.038 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op12, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.062 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/MatMul-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.084 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/gradMatMul-expand/MatMul-op6, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.102 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 add input node monad device tensor store:fc1.weight [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.114 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 add input node monad device tensor store:moments.fc1.weight [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.125 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.135 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.183 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op13, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.206 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/Conv2D-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.234 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Gradients/Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/gradConv2D-expand/Conv2DBackpropInput-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.251 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 add input node monad device tensor store:conv2.weight [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.262 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 add input node monad device tensor store:moments.conv2.weight [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.273 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.283 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.353 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op14, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.376 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1965] LinkControlArrowByAutoMonad] Link control arrow by auto monad from actor: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv1-Conv2d/Conv2D-op1, to actor: Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 for the graph: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.395 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 add input node monad device tensor store:conv1.weight [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.408 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 add input node monad device tensor store:moments.conv1.weight [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.419 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 add input node monad device tensor store:learning_rate [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.430 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:191] operator()] Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15, kernel:Default/network-TrainOneStepCell/optimizer-Momentum/ApplyMomentum-op15 add input node monad device tensor store:momentum [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.449 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op14 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.472 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op11 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.494 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op8 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.515 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op12 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.534 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op15 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.553 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op10 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.571 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op13 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.624.589 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2012] LinkControlArrowBySendRecvNodes] Link control arrow for parallel node output: Default/network-TrainOneStepCell/grad_reducer-DistributedGradReducer/AllReduce-op9 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.625.350 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 1_actor_set_kernel_graph_1_memory_actor_insert in 19.07 us [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.625.385 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 2_actor_set_kernel_graph_1_invalid_data_arrow_elimination in 6.25 us [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.625.610 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 3_actor_set_kernel_graph_1_multi_actor_fusion in 202.25 us [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.625.653 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 4_actor_set_kernel_graph_1_batch_data_arrow_fusion in 19.34 us [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:37:32.625.671 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:554] Transform] Graph(kernel_graph_1) transforms actor end. [WARNING] VM(164046,ffff82e63440,python):2024-01-10-11:37:32.627.075 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:612] CompileGraphs] [PROF]compile_func_graph costs 86080 usec. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:37:32.627.120 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:615] CompileGraphs] Status record: end compile function graph: 381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316, produce actor: kernel_graph_1 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.627.153 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end task_emit action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.627.176 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:268] SetLoopCount] Change vm_loop_flag to 0, set loop_size to 468 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.627.194 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start execute action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.627.214 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end execute action. TotalTime = 0.753852, [19] [parse]: 0.0116798 [symbol_resolve]: 0.242797, [1] [Cycle 1]: 0.242161, [1] [resolve]: 0.242135 [graph_reusing]: 7.265e-05 [meta_unpack_prepare]: 0.0010549 [pre_cconv]: 3.183e-05 [abstract_specialize]: 0.238957 [pack_expand]: 0.00046086 [auto_monad]: 0.00325827 [inline]: 3.629e-05 [pre_auto_parallel]: 6.073e-05 [pipeline_split]: 3.932e-05 [optimize]: 0.16617, [35] [py_interpret_to_execute]: 0.00115196 [rewriter_before_opt_a]: 0.00528525 [opt_a]: 0.15267, [3] [Cycle 1]: 0.122357, [30] [expand_dump_flag]: 6.42e-05 [switch_simplify]: 0.00102038 [a_1]: 0.0169612 [recompute_prepare]: 0.0001493 [updatestate_depend_eliminate]: 0.00080619 [updatestate_assign_eliminate]: 0.00021742 [updatestate_loads_eliminate]: 0.00046396 [parameter_eliminate]: 2.048e-05 [a_2]: 0.00274874 [accelerated_algorithm]: 8.588e-05 [pynative_shard]: 5.435e-05 [auto_parallel]: 7.37001e-06 [parallel]: 4.721e-05 [merge_comm]: 7.572e-05 [allreduce_fusion]: 3.813e-05 [virtual_dataset]: 8.266e-05 [get_grad_eliminate_]: 6.534e-05 [virtual_output]: 6.391e-05 [merge_forward]: 0.00011668 [cell_reuse_recompute_pass]: 8.2e-07 [cell_reuse_handle_not_recompute_node_pass]: 0.0001753 [meta_fg_expand]: 0.0344014 [after_resolve]: 0.00038041 [a_after_grad]: 0.00058182 [renormalize]: 0.053402 [real_op_eliminate]: 0.0006398 [auto_monad_grad]: 0.00047849 [auto_monad_eliminator]: 0.00084962 [cse]: 0.00238932 [a_3]: 0.00562787 [Cycle 2]: 0.021176, [30] [expand_dump_flag]: 2.487e-05 [switch_simplify]: 0.00030429 [a_1]: 0.00830613 [recompute_prepare]: 5.661e-05 [updatestate_depend_eliminate]: 0.00053273 [updatestate_assign_eliminate]: 7.785e-05 [updatestate_loads_eliminate]: 0.00015825 [parameter_eliminate]: 3.09e-06 [a_2]: 0.00089132 [accelerated_algorithm]: 6.94e-05 [pynative_shard]: 5.635e-05 [auto_parallel]: 8.1e-06 [parallel]: 9.27e-06 [merge_comm]: 4.14e-05 [allreduce_fusion]: 2.471e-05 [virtual_dataset]: 5.154e-05 [get_grad_eliminate_]: 4.443e-05 [virtual_output]: 4.304e-05 [merge_forward]: 7.056e-05 [cell_reuse_recompute_pass]: 6.90001e-07 [cell_reuse_handle_not_recompute_node_pass]: 0.00011355 [meta_fg_expand]: 8.896e-05 [after_resolve]: 5.876e-05 [a_after_grad]: 5.531e-05 [renormalize]: 0.00829866 [real_op_eliminate]: 6.906e-05 [auto_monad_grad]: 4.11e-06 [auto_monad_eliminator]: 0.00026593 [cse]: 0.00084115 [a_3]: 0.00039139 [Cycle 3]: 0.00362227, [30] [expand_dump_flag]: 1.9e-06 [switch_simplify]: 4.771e-05 [a_1]: 0.00068548 [recompute_prepare]: 4.3e-05 [updatestate_depend_eliminate]: 0.00010245 [updatestate_assign_eliminate]: 6.721e-05 [updatestate_loads_eliminate]: 6.787e-05 [parameter_eliminate]: 1.72e-06 [a_2]: 0.00091034 [accelerated_algorithm]: 6.978e-05 [pynative_shard]: 4.49e-05 [auto_parallel]: 5.53e-06 [parallel]: 6.82e-06 [merge_comm]: 3.888e-05 [allreduce_fusion]: 1.885e-05 [virtual_dataset]: 5.298e-05 [get_grad_eliminate_]: 4.638e-05 [virtual_output]: 4.508e-05 [merge_forward]: 6.497e-05 [cell_reuse_recompute_pass]: 4.69998e-07 [cell_reuse_handle_not_recompute_node_pass]: 0.00012079 [meta_fg_expand]: 6.181e-05 [after_resolve]: 6.058e-05 [a_after_grad]: 5.866e-05 [renormalize]: 7.0002e-08 [real_op_eliminate]: 4.665e-05 [auto_monad_grad]: 2.09e-06 [auto_monad_eliminator]: 0.00014827 [cse]: 0.00026283 [a_3]: 0.00038491 [py_interpret_to_execute_after_opt_a]: 9.275e-05 [slice_cell_reuse_recomputed_activation]: 3.03e-06 [rewriter_after_opt_a]: 0.00189626 [convert_after_rewriter]: 8.504e-05 [order_py_execute_after_rewriter]: 5.832e-05 [opt_b]: 0.00187607, [1] [Cycle 1]: 0.001869, [7] [b_1]: 0.00126141 [b_2]: 5.298e-05 [updatestate_depend_eliminate]: 7.018e-05 [updatestate_assign_eliminate]: 6.592e-05 [updatestate_loads_eliminate]: 7.178e-05 [renormalize]: 4.294e-05 [cse]: 0.00025635 [cconv]: 5.562e-05 [opt_after_cconv]: 0.00073417, [1] [Cycle 1]: 0.00072724, [7] [c_1]: 0.00017492 [parameter_eliminate]: 1.63e-06 [updatestate_depend_eliminate]: 8.407e-05 [updatestate_assign_eliminate]: 6.856e-05 [updatestate_loads_eliminate]: 7.163e-05 [cse]: 0.00024615 [renormalize]: 3.939e-05 [remove_dup_value]: 0.0003032 [tuple_transform]: 0.00050265, [1] [Cycle 1]: 0.00049608, [3] [d_1]: 0.00030431 [d_2]: 0.00013397 [renormalize]: 3.663e-05 [add_cache_embedding]: 0.00014144 [add_recomputation]: 0.00059428 [cse_after_recomputation]: 0.00025159, [1] [Cycle 1]: 0.00024474, [1] [cse]: 0.00023477 [environ_conv]: 8.118e-05 [label_micro_interleaved_index]: 2.69e-06 [label_fine_grained_interleaved_index]: 2.27e-06 [assign_add_opt]: 3.949e-05 [slice_recompute_activation]: 2.81e-06 [micro_interleaved_order_control]: 1.9e-06 [full_micro_interleaved_order_control]: 2.35e-06 [comp_comm_scheduling]: 2.16001e-06 [reorder_send_recv_between_fp_bp]: 1.86e-06 [comm_op_add_attrs]: 1.06001e-06 [add_comm_op_reuse_tag]: 1.956e-05 [overlap_opt_shard_in_pipeline]: 1.412e-05 [grouped_pairwise_exchange_alltoall]: 1.41e-05 [overlap_recompute_and_grad_model_parallel]: 1.77e-06 [overlap_grad_matmul_and_grad_allreduce]: 1.263e-05 [split_matmul_comm_elemetwise]: 1.387e-05 [split_layernorm_comm]: 1.8e-06 [process_send_recv_for_ge]: 2.68e-06 [handle_group_info]: 7.30004e-07 [auto_monad_reorder]: 0.00035798 [get_jit_bprop_graph]: 2.687e-05 [eliminate_special_op_node]: 0.00110124 [validate]: 0.00031737 [distribtued_split]: 0.00044691 [task_emit]: 0.0866563 [execute]: 3.079e-05 Sums parse : 0.011680s : 1.57% symbol_resolve.resolve : 0.242135s : 32.45% graph_reusing : 0.000073s : 0.01% meta_unpack_prepare : 0.001055s : 0.14% pre_cconv : 0.000032s : 0.00% abstract_specialize : 0.238957s : 32.02% pack_expand : 0.000461s : 0.06% auto_monad : 0.003258s : 0.44% inline : 0.000036s : 0.00% pre_auto_parallel : 0.000061s : 0.01% pipeline_split : 0.000039s : 0.01% optimize.py_interpret_to_execute : 0.001152s : 0.15% optimize.rewriter_before_opt_a : 0.005285s : 0.71% optimize.opt_a.expand_dump_flag : 0.000091s : 0.01% optimize.opt_a.switch_simplify : 0.001372s : 0.18% optimize.opt_a.a_1 : 0.025953s : 3.48% optimize.opt_a.recompute_prepare : 0.000249s : 0.03% optimize.opt_a.updatestate_depend_eliminate : 0.001441s : 0.19% optimize.opt_a.updatestate_assign_eliminate : 0.000362s : 0.05% optimize.opt_a.updatestate_loads_eliminate : 0.000690s : 0.09% optimize.opt_a.parameter_eliminate : 0.000025s : 0.00% optimize.opt_a.a_2 : 0.004550s : 0.61% optimize.opt_a.accelerated_algorithm : 0.000225s : 0.03% optimize.opt_a.pynative_shard : 0.000156s : 0.02% optimize.opt_a.auto_parallel : 0.000021s : 0.00% optimize.opt_a.parallel : 0.000063s : 0.01% optimize.opt_a.merge_comm : 0.000156s : 0.02% optimize.opt_a.allreduce_fusion : 0.000082s : 0.01% optimize.opt_a.virtual_dataset : 0.000187s : 0.03% optimize.opt_a.get_grad_eliminate_ : 0.000156s : 0.02% optimize.opt_a.virtual_output : 0.000152s : 0.02% optimize.opt_a.merge_forward : 0.000252s : 0.03% optimize.opt_a.cell_reuse_recompute_pass : 0.000002s : 0.00% optimize.opt_a.cell_reuse_handle_not_recompute_node_pass : 0.000410s : 0.05% optimize.opt_a.meta_fg_expand : 0.034552s : 4.63% optimize.opt_a.after_resolve : 0.000500s : 0.07% optimize.opt_a.a_after_grad : 0.000696s : 0.09% optimize.opt_a.renormalize : 0.061701s : 8.27% optimize.opt_a.real_op_eliminate : 0.000756s : 0.10% optimize.opt_a.auto_monad_grad : 0.000485s : 0.06% optimize.opt_a.auto_monad_eliminator : 0.001264s : 0.17% optimize.opt_a.cse : 0.003493s : 0.47% optimize.opt_a.a_3 : 0.006404s : 0.86% optimize.py_interpret_to_execute_after_opt_a : 0.000093s : 0.01% optimize.slice_cell_reuse_recomputed_activation : 0.000003s : 0.00% optimize.rewriter_after_opt_a : 0.001896s : 0.25% optimize.convert_after_rewriter : 0.000085s : 0.01% optimize.order_py_execute_after_rewriter : 0.000058s : 0.01% optimize.opt_b.b_1 : 0.001261s : 0.17% optimize.opt_b.b_2 : 0.000053s : 0.01% optimize.opt_b.updatestate_depend_eliminate : 0.000070s : 0.01% optimize.opt_b.updatestate_assign_eliminate : 0.000066s : 0.01% optimize.opt_b.updatestate_loads_eliminate : 0.000072s : 0.01% optimize.opt_b.renormalize : 0.000043s : 0.01% optimize.opt_b.cse : 0.000256s : 0.03% optimize.cconv : 0.000056s : 0.01% optimize.opt_after_cconv.c_1 : 0.000175s : 0.02% optimize.opt_after_cconv.parameter_eliminate : 0.000002s : 0.00% optimize.opt_after_cconv.updatestate_depend_eliminate : 0.000084s : 0.01% optimize.opt_after_cconv.updatestate_assign_eliminate : 0.000069s : 0.01% optimize.opt_after_cconv.updatestate_loads_eliminate : 0.000072s : 0.01% optimize.opt_after_cconv.cse : 0.000246s : 0.03% optimize.opt_after_cconv.renormalize : 0.000039s : 0.01% optimize.remove_dup_value : 0.000303s : 0.04% optimize.tuple_transform.d_1 : 0.000304s : 0.04% optimize.tuple_transform.d_2 : 0.000134s : 0.02% optimize.tuple_transform.renormalize : 0.000037s : 0.00% optimize.add_cache_embedding : 0.000141s : 0.02% optimize.add_recomputation : 0.000594s : 0.08% optimize.cse_after_recomputation.cse : 0.000235s : 0.03% optimize.environ_conv : 0.000081s : 0.01% optimize.label_micro_interleaved_index : 0.000003s : 0.00% optimize.label_fine_grained_interleaved_index : 0.000002s : 0.00% optimize.assign_add_opt : 0.000039s : 0.01% optimize.slice_recompute_activation : 0.000003s : 0.00% optimize.micro_interleaved_order_control : 0.000002s : 0.00% optimize.full_micro_interleaved_order_control : 0.000002s : 0.00% optimize.comp_comm_scheduling : 0.000002s : 0.00% optimize.reorder_send_recv_between_fp_bp : 0.000002s : 0.00% optimize.comm_op_add_attrs : 0.000001s : 0.00% optimize.add_comm_op_reuse_tag : 0.000020s : 0.00% optimize.overlap_opt_shard_in_pipeline : 0.000014s : 0.00% optimize.grouped_pairwise_exchange_alltoall : 0.000014s : 0.00% optimize.overlap_recompute_and_grad_model_parallel : 0.000002s : 0.00% optimize.overlap_grad_matmul_and_grad_allreduce : 0.000013s : 0.00% optimize.split_matmul_comm_elemetwise : 0.000014s : 0.00% optimize.split_layernorm_comm : 0.000002s : 0.00% optimize.process_send_recv_for_ge : 0.000003s : 0.00% optimize.handle_group_info : 0.000001s : 0.00% auto_monad_reorder : 0.000358s : 0.05% get_jit_bprop_graph : 0.000027s : 0.00% eliminate_special_op_node : 0.001101s : 0.15% validate : 0.000317s : 0.04% distribtued_split : 0.000447s : 0.06% task_emit : 0.086656s : 11.61% execute : 0.000031s : 0.00% Time group info: ------[substitution.] 0.251595 4489 0.08% : 0.000210s : 57: substitution.arithmetic_simplify 0.02% : 0.000040s : 16: substitution.cast_eliminate 0.02% : 0.000042s : 55: substitution.depend_value_elim 0.01% : 0.000017s : 16: substitution.environ_get_add_eliminate 0.00% : 0.000007s : 8: substitution.environ_get_depend_swap 0.01% : 0.000021s : 32: substitution.environ_get_eliminate 0.02% : 0.000038s : 16: substitution.environ_get_set_eliminate 0.01% : 0.000038s : 94: substitution.float_depend_g_call 0.00% : 0.000007s : 16: substitution.float_environ_get_switch 0.00% : 0.000013s : 14: substitution.float_tuple_getitem_switch 94.68% : 0.238201s : 233: substitution.getattr_setattr_resolve 0.02% : 0.000052s : 120: substitution.graph_param_transform 0.00% : 0.000006s : 20: substitution.incorporate_call 0.00% : 0.000004s : 20: substitution.incorporate_call_switch 4.10% : 0.010305s : 377: substitution.inline 0.01% : 0.000013s : 34: substitution.less_batch_normalization 0.01% : 0.000029s : 160: substitution.load_eliminater 0.15% : 0.000368s : 505: substitution.meta_unpack_prepare 0.02% : 0.000055s : 72: substitution.minmaximum_grad 0.01% : 0.000017s : 8: substitution.partial_defer_inline 0.05% : 0.000131s : 94: substitution.partial_eliminate 0.00% : 0.000009s : 120: substitution.partial_unused_args_eliminate 0.02% : 0.000042s : 31: substitution.real_op_eliminate 0.00% : 0.000012s : 32: substitution.reduce_all_const_elim 0.01% : 0.000032s : 338: substitution.remove_not_recompute_node 0.12% : 0.000296s : 264: substitution.replace_applicator 0.01% : 0.000030s : 142: substitution.replace_old_param 0.00% : 0.000011s : 2: substitution.reshape_eliminate 0.00% : 0.000006s : 10: substitution.set_cell_output_no_recompute 0.00% : 0.000006s : 2: substitution.specialize_transform 0.01% : 0.000020s : 32: substitution.split_environ_get_set_with_tuple_value 0.01% : 0.000024s : 31: substitution.switch_simplify 0.07% : 0.000185s : 76: substitution.tuple_list_convert_item_index_to_positive 0.03% : 0.000083s : 92: substitution.tuple_list_get_item_const_eliminator 0.06% : 0.000160s : 92: substitution.tuple_list_get_item_depend_reorder 0.19% : 0.000466s : 283: substitution.tuple_list_get_item_eliminator 0.05% : 0.000117s : 92: substitution.tuple_list_get_set_item_eliminator 0.08% : 0.000195s : 416: substitution.updatestate_pure_node_eliminater 0.11% : 0.000285s : 467: substitution.updatestate_useless_node_eliminater ------[renormalize.] 0.061681 4 58.58% : 0.036133s : 2: renormalize.infer 41.42% : 0.025548s : 2: renormalize.specialize ------[replace.] 0.007749 886 0.07% : 0.000006s : 1: replace.arithmetic_simplify 1.06% : 0.000082s : 16: replace.cast_eliminate 0.66% : 0.000051s : 10: replace.depend_value_elim 1.03% : 0.000079s : 8: replace.environ_get_set_eliminate 33.19% : 0.002572s : 210: replace.getattr_setattr_resolve 31.62% : 0.002450s : 341: replace.inline 0.34% : 0.000026s : 1: replace.meta_unpack_prepare 5.78% : 0.000448s : 32: replace.partial_eliminate 1.88% : 0.000145s : 31: replace.real_op_eliminate 1.93% : 0.000149s : 9: replace.replace_applicator 3.62% : 0.000280s : 31: replace.switch_simplify 1.35% : 0.000105s : 16: replace.tuple_list_get_item_depend_reorder 17.23% : 0.001335s : 179: replace.tuple_list_get_item_eliminator 0.26% : 0.000020s : 1: replace.updatestate_useless_node_eliminater ------[match.] 0.248526 886 0.00% : 0.000012s : 1: match.arithmetic_simplify 0.02% : 0.000040s : 16: match.cast_eliminate 0.00% : 0.000007s : 10: match.depend_value_elim 0.01% : 0.000032s : 8: match.environ_get_set_eliminate 95.62% : 0.237633s : 210: match.getattr_setattr_resolve 4.06% : 0.010102s : 341: match.inline 0.06% : 0.000148s : 1: match.meta_unpack_prepare 0.04% : 0.000093s : 32: match.partial_eliminate 0.02% : 0.000042s : 31: match.real_op_eliminate 0.01% : 0.000032s : 9: match.replace_applicator 0.01% : 0.000024s : 31: match.switch_simplify 0.02% : 0.000055s : 16: match.tuple_list_get_item_depend_reorder 0.12% : 0.000300s : 179: match.tuple_list_get_item_eliminator 0.00% : 0.000006s : 1: match.updatestate_useless_node_eliminater ------[func_graph_cloner_run.] 0.040415 648 70.65% : 0.028555s : 267: func_graph_cloner_run.FuncGraphClonerGraph 29.35% : 0.011860s : 381: func_graph_cloner_run.FuncGraphSpecializer ------[meta_graph.] 0.000000 0 ------[manager.] 0.000000 0 ------[pynative] 0.000000 0 ------[others.] 0.287849 170 3.92% : 0.011281s : 103: opt.transform.opt_a 0.43% : 0.001245s : 23: opt.transform.opt_b 84.53% : 0.243327s : 6: opt.transform.opt_resolve 0.34% : 0.000969s : 1: opt.transforms.meta_unpack_prepare 10.49% : 0.030202s : 30: opt.transforms.opt_a 0.06% : 0.000172s : 1: opt.transforms.opt_after_cconv 0.02% : 0.000051s : 1: opt.transforms.opt_b 0.15% : 0.000434s : 2: opt.transforms.opt_trans_graph 0.06% : 0.000170s : 3: opt.transforms.special_op_eliminate [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.627.931 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1385] Run] End [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.627.961 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:846] SaveCompiledGraph] Save compiled func graph(381_251_1_mindspore_train_dataset_helper__DataWrapper_construct_316) phase(train.1704857851709617664.281469736529552.0)! [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.627.981 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:864] SaveCompiledGraph] End save compiled func graph! [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.627.994 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:942] CleanCompileRes] Clean compile resource start [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.638.098 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:956] CleanCompileRes] Clean compile resource end [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.638.131 [mindspore/ccsrc/pipeline/jit/ps/event_message_print.cc:37] PrintEventMessage] End compiling '_DataWrapper.construct'. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.638.147 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1039] CompileInner] Finish compiling. [WARNING] ME(164046:281472877868096,MainProcess):2024-01-10-11:37:32.639.184 [mindspore/parallel/_utils.py:259] You are suggested to use mindspore.context.set_auto_parallel_context(parameter_broadcast=True) or mindspore.common.set_seed() to share parameters among multi-devices. [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:37:32.641.640 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:37:32.641.865 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:37:32.641.908 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_1 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.642.122 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[1], dtype=Int64, value=[32]), output index: 0 device address:0x4a9c6b10 [INFO] PRE_ACT(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.642.192 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:237] SetMemAllocUintSize] Set mem alloc unit size, common 1073741824 persistent 1073741824 [INFO] DEVICE(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.642.213 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_pool.cc:124] AllocDeviceMem] Malloc Memory for Pool, size: 1073741824 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.642.425 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[32, 1], dtype=Float32, value=[...]), output index: 0 device address:0x4a986d70 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.642.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), output index: 0 device address:0x4a8ce2e0 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.642.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode (16, 6, 5, 5) [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.642.686 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10]), output index: 0 device address:0x4a2b5240 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.642.757 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode true [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.642.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10]), output index: 0 device address:0x4dd059b0 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.642.919 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode (32, 16, 5, 5) [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.643.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode false [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.643.086 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[1], dtype=Int32, value=[1]), output index: 0 device address:0x4dc2c410 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.643.155 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode -1 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.643.236 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode (6, 1, 5, 5) [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.643.308 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode (32, 6, 14, 14) [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.643.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Float32, value=0), output index: 0 device address:0x4dee8730 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.643.465 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Int64, value=10), output index: 0 device address:0x4dd8ca70 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.643.532 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode 0 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.643.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode 1 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.643.677 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Float32, value=1), output index: 0 device address:0x4dd8eeb0 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.643.755 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Float32, value=0.25), output index: 0 device address:0x4dcf7140 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.643.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc3.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.643.949 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc3.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.644.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc2.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.644.149 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc2.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.644.252 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc1.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.644.344 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_fc1.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.644.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_conv2.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.644.563 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_conv1.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.644.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_global_step, device type:2 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.644.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc3.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.644.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_learning_rate, device type:2 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.644.925 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_momentum, device type:2 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.645.016 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc3.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.645.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc2.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.645.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc2.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.645.289 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc1.bias, device type:2 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.645.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.fc1.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.645.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.conv2.weight, device type:2 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.645.617 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:1085] PrepareDataForWeightNode] Prepare device data for weight node:@kernel_graph_1:param_moments.conv1.weight, device type:2 [INFO] PRE_ACT(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.645.767 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:237] SetMemAllocUintSize] Set mem alloc unit size, common 1073741824 persistent 1073741824 [INFO] DEVICE(164046,fffe1bf4f0f0,python):2024-01-10-11:37:32.645.792 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_pool.cc:124] AllocDeviceMem] Malloc Memory for Pool, size: 1073741824 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:37:32.645.828 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] GE(164046,python):2024-01-10-11:37:32.724.403 [graph_var_manager.cc:1424][EVENT]167353 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:32.724.489 [graph_manager.cc:1248][EVENT]167353 PreRun:PreRun start: graph node size 2, session id 2, graph id 1, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:32.725.347 [atrace_api.c:28](tid:167353) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:32.725.419 [trace_rb_log.c:84](tid:167353) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:32.725.433 [atrace_api.c:32](tid:167353) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:32.725.452 [client_manager.cpp:157][SetProfilingCallback][tid:167353] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:32.726.344 [parallel_partitioner.cc:165][EVENT]167353 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.726.385 [parallel_partitioner.cc:178][EVENT]167353 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.726.427 [graph_prepare.cc:1378][EVENT]167353 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.727.090 [graph_manager.cc:1050][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [677] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.727.120 [graph_manager.cc:1052][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.727.190 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.727.218 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.727.300 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [71] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.727.315 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.727.372 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.727.385 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.727.395 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.727.519 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.727.541 [graph_manager.cc:1054][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [408] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.727.786 [graph_manager.cc:1055][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [231] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.728.661 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.728.689 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.728.700 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.728.710 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [261] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.728.719 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.728.728 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.728.737 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [44] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.728.745 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.728.754 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.139 [graph_manager.cc:1056][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2332] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.730.203 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.221 [graph_prepare.cc:1982][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [47] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.730.564 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.730.588 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.599 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.608 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [129] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.617 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [6] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.626 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.730.635 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.644 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [6] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.662 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.700 [graph_prepare.cc:1983][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [465] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.730.724 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.730.735 [graph_prepare.cc:1984][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.730.750 [graph_prepare.cc:1985][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.730.768 [graph_prepare.cc:1986][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.730.780 [graph_prepare.cc:1987][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.730.795 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.730.806 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.730.820 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.730.889 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.901 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.910 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.919 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.927 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.936 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.944 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.953 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.961 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.970 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.978 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [0] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.987 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.730.995 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.731.011 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.731.021 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.731.029 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.731.051 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.731.065 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.731.093 [graph_prepare.cc:1988][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [304] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.731.105 [graph_manager.cc:1065][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [932] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.744.197 [graph_manager.cc:1077][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13072] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.744.264 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.744.316 [graph_manager.cc:1080][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [82] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.058 [graph_manager.cc:1081][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [2726] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.098 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.112 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.124 [graph_manager.cc:1082][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [32] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.152 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.166 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.180 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.213 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [24] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.227 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.239 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.252 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.296 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [26] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.314 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.350 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [25] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.373 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.387 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.399 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.408 [graph_manager.cc:2700][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [260] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.486 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.747.499 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.747.508 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.747.517 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.747.526 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.747.534 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CastRemovePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.747.543 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.747.552 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.747.560 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.747.569 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.747.577 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [6] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.747.585 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.747.594 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [6] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.747.602 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.747.610 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.747.620 [graph_manager.cc:2741][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [195] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.636 [graph_manager.cc:2752][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.658 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.669 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.684 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.699 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.710 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.722 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.740 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.754 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.766 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.777 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.795 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.807 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.824 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.836 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.845 [graph_manager.cc:2810][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [190] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.868 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.747.880 [graph_manager.cc:2821][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [27] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.747.907 [graph_manager.cc:1087][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [766] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.748.041 [graph_manager.cc:1088][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [121] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.748.078 [graph_manager.cc:1089][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.748.095 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.748.121 [graph_manager.cc:1097][EVENT]167353 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:32.748.142 [graph_manager.cc:3325][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.748.268 [engine_place.cc:144][EVENT]167353 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.748.283 [engine_place.cc:144][EVENT]167353 Run:The time cost of aicpu_ascend_kernel::CheckSupported is [43] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.748.347 [graph_manager.cc:3351][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [191] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.748.364 [graph_manager.cc:3364][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.748.418 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.748.433 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.748.545 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [102] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.748.571 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.748.608 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [26] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.748.637 [graph_manager.cc:3405][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [262] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.748.654 [graph_manager.cc:3412][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.750.688 [graph_manager.cc:3422][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [2018] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.750.720 [graph_manager.cc:3428][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.750.834 [graph_manager.cc:3467][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [94] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.750.852 [graph_manager.cc:3377][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [2476] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.750.866 [graph_manager.cc:1106][EVENT]167353 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [2731] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.750.878 [graph_manager.cc:1115][EVENT]167353 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:32.750.900 [graph_manager.cc:1130][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.750.941 [graph_manager.cc:1131][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.750.963 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.750.978 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.750.988 [graph_manager.cc:2837][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [32] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.063 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [8] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.751.075 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.751.084 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.751.093 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.751.102 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [21] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.751.111 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:37:32.751.121 [graph_manager.cc:2864][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [117] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.132 [graph_manager.cc:2872][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.150 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.163 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.177 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.189 [compile_nodes_pass.cc:88][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.199 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.210 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.287 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [68] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.310 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.323 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.336 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.355 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.364 [graph_manager.cc:2927][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [217] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.376 [graph_manager.cc:2937][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.404 [graph_manager.cc:2943][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.418 [graph_manager.cc:2950][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.612 [graph_manager.cc:2958][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.642 [graph_manager.cc:1132][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [687] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.749 [graph_manager.cc:1135][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [94] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.785 [graph_manager.cc:2975][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.885 [graph_manager.cc:2981][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [88] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.901 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.912 [graph_manager.cc:2986][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.751.922 [graph_manager.cc:1136][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [157] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.752.011 [graph_manager.cc:3555][EVENT]167353 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [65] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.752.063 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.752.077 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.752.169 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [82] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.752.190 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.752.221 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.752.241 [graph_builder.cc:865][EVENT]167353 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [202] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.752.350 [graph_builder.cc:288][EVENT]167353 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [79] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.752.532 [graph_builder.cc:293][EVENT]167353 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [167] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.752.717 [model_builder.cc:1133][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [94] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.752.957 [block_mem_assigner.cc:4069][EVENT]167660 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164046,python):2024-01-10-11:37:32.752.960 [block_mem_assigner.cc:4069][EVENT]167661 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164046,python):2024-01-10-11:37:32.753.410 [graph_mem_assigner.cc:2166][EVENT]167353 SetInputOffset:[IMAS]AfterAssignMemory : online_1 memoffset[132096], memtype[2], theory_min[264192], zero_copy[132096], total_size[132096], no_reuse[132096], streams[1], topo_mode[DFS], mop[], io_reuse[0:0], alloc_mode[] [INFO] GE(164046,python):2024-01-10-11:37:32.753.494 [model_builder.cc:1144][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [755] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.753.517 [model_builder.cc:1152][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.753.531 [model_builder.cc:1157][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.753.635 [model_builder.cc:1167][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [93] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.753.653 [model_builder.cc:1174][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.753.673 [model_builder.cc:1180][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.753.722 [model_builder.cc:1184][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.753.743 [graph_builder.cc:304][EVENT]167353 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [1189] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:32.753.974 [logger.cc:1071] 167353 ModelBindStream: model_id=64, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:37:32.754.038 [task_generator.cc:804][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.754.094 [task_generator.cc:805][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [41] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.754.599 [task_generator.cc:814][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [491] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.754.613 [task_generator.cc:954][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [581] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.754.667 [task_generator.cc:967][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [30] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:32.754.685 [logger.cc:1084] 167353 ModelUnbindStream: model_id=64, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:37:32.754.737 [graph_builder.cc:310][EVENT]167353 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [981] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.754.842 [graph_manager.cc:1152][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2901] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.754.859 [graph_manager.cc:1164][EVENT]167353 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:32.754.889 [graph_manager.cc:1271][EVENT]167353 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [28627] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.754.900 [graph_manager.cc:1272][EVENT]167353 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:37:32.755.204 [atrace_api.c:93](tid:167353) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:37:32.755.220 [atrace_api.c:95](tid:167353) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:37:32.755.878 [model_introduction.cc:236][EVENT]167353 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(164046,python):2024-01-10-11:37:32.755.899 [model_introduction.cc:294][EVENT]167353 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(164046,python):2024-01-10-11:37:32.755.913 [model_introduction.cc:366][EVENT]167353 Data:model io_info size:114 [INFO] GE(164046,python):2024-01-10-11:37:32.759.308 [graph_converter.cc:838][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1317] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.759.530 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [176] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.759.951 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [394] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.760.039 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [60] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.760.058 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [82] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.760.101 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [30] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.760.133 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.760.163 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.760.232 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [57] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.760.295 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [49] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.760.308 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [62] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.760.339 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.760.366 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.760.382 [graph_converter.cc:849][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1034] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.760.581 [graph_converter.cc:853][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [188] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.761.192 [graph_converter.cc:857][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [584] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.761.305 [graph_converter.cc:862][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [86] micro second. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:37:32.765.937 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 101 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] GE(164046,python):2024-01-10-11:37:32.841.056 [graph_var_manager.cc:1424][EVENT]167352 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:32.841.149 [graph_manager.cc:1248][EVENT]167352 PreRun:PreRun start: graph node size 6, session id 3, graph id 2, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:32.841.487 [atrace_api.c:28](tid:167352) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:32.841.517 [trace_rb_log.c:84](tid:167352) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:32.841.530 [atrace_api.c:32](tid:167352) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:32.841.549 [client_manager.cpp:157][SetProfilingCallback][tid:167352] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:32.842.046 [parallel_partitioner.cc:165][EVENT]167352 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.842.088 [parallel_partitioner.cc:178][EVENT]167352 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.842.135 [graph_prepare.cc:1378][EVENT]167352 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.842.301 [graph_manager.cc:1050][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [183] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.842.324 [graph_manager.cc:1052][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.842.499 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.842.529 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.842.575 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.842.588 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.842.632 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.842.646 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.842.664 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.842.770 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.842.791 [graph_manager.cc:1054][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [453] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.843.048 [graph_manager.cc:1055][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [222] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.844.461 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:37:32.844.491 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.844.503 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.844.513 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [455] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.844.523 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.844.532 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:37:32.844.541 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [68] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.844.550 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.844.559 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [7] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.847.029 [graph_manager.cc:1056][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3961] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.847.104 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.847.123 [graph_prepare.cc:1982][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [58] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.847.679 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:37:32.847.706 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.847.717 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.847.727 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [334] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.847.736 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.847.745 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:37:32.847.754 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.847.763 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.847.772 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.847.811 [graph_prepare.cc:1983][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [674] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.847.836 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.847.848 [graph_prepare.cc:1984][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.847.862 [graph_prepare.cc:1985][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.847.876 [graph_prepare.cc:1986][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.847.888 [graph_prepare.cc:1987][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.847.902 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.847.916 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.847.932 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.848.044 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.848.059 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.848.068 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.848.077 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.848.086 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.848.095 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.848.103 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.848.112 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.848.121 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.848.129 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.848.137 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.848.146 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.848.154 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.848.162 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [9] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.848.179 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.848.188 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of IdentityPass is [5] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.848.211 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.848.225 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.848.260 [graph_prepare.cc:1988][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [362] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.848.272 [graph_manager.cc:1065][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1207] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.861.323 [graph_manager.cc:1077][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13030] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.861.393 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.861.442 [graph_manager.cc:1080][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [82] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.869.330 [graph_manager.cc:1081][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [7872] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.869.373 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.869.392 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.869.406 [graph_manager.cc:1082][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [40] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.869.439 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.869.455 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.869.470 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.869.501 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.869.518 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.869.538 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.869.554 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.869.597 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [34] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.869.631 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.869.665 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.869.710 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [33] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.869.730 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.869.744 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.869.755 [graph_manager.cc:2700][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [322] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.869.897 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.869.913 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.869.923 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.869.932 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.869.941 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.869.949 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CastRemovePass is [12] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.869.958 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.869.967 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.869.975 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [5] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.869.983 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.869.992 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.870.000 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.870.009 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.870.017 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.870.026 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.870.036 [graph_manager.cc:2741][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [260] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.045 [graph_manager.cc:2752][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.075 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.087 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.106 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.121 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.133 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.145 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.165 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.178 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.191 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.201 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.214 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.226 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.245 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.257 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.266 [graph_manager.cc:2810][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [194] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.300 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.870.311 [graph_manager.cc:2821][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.338 [graph_manager.cc:1087][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [912] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.476 [graph_manager.cc:1088][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [125] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.520 [graph_manager.cc:1089][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.538 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.870.553 [graph_manager.cc:1097][EVENT]167352 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:32.870.586 [graph_manager.cc:3325][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.871.745 [engine_place.cc:144][EVENT]167352 Run:The time cost of AIcoreEngine::CheckSupported is [1020] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.871.774 [engine_place.cc:144][EVENT]167352 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.871.785 [engine_place.cc:144][EVENT]167352 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.871.936 [graph_manager.cc:3351][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [1335] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.871.955 [graph_manager.cc:3364][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.872.022 [engine_partitioner.cc:1139][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.872.040 [engine_partitioner.cc:1142][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.872.235 [engine_partitioner.cc:1148][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [185] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.872.282 [engine_partitioner.cc:1155][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [33] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.872.329 [engine_partitioner.cc:1164][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.872.364 [graph_manager.cc:3405][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [396] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.872.382 [graph_manager.cc:3412][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.227 [graph_manager.cc:3422][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [10831] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.272 [graph_manager.cc:3428][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.430 [graph_manager.cc:3467][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [134] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.449 [graph_manager.cc:3377][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [11481] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.466 [graph_manager.cc:1106][EVENT]167352 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [12885] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.479 [graph_manager.cc:1115][EVENT]167352 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:32.883.506 [graph_manager.cc:1130][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.542 [graph_manager.cc:1131][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.582 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.601 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.611 [graph_manager.cc:2837][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.703 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.883.715 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.883.725 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.883.734 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.883.743 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [7] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.883.751 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:32.883.761 [graph_manager.cc:2864][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [132] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.773 [graph_manager.cc:2872][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.791 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.805 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.821 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.835 [compile_nodes_pass.cc:88][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.848 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.858 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.950 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [82] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.983 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.883.997 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.884.011 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.884.031 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.884.041 [graph_manager.cc:2927][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [252] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.884.053 [graph_manager.cc:2937][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.884.068 [graph_manager.cc:2943][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.884.079 [graph_manager.cc:2950][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.884.277 [graph_manager.cc:2958][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [44] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.884.312 [graph_manager.cc:1132][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [740] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.884.388 [graph_manager.cc:1135][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [61] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.884.423 [graph_manager.cc:2975][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.884.454 [graph_manager.cc:2981][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.884.469 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.884.479 [graph_manager.cc:2986][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.884.489 [graph_manager.cc:1136][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [83] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.884.623 [graph_manager.cc:3555][EVENT]167352 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [98] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.884.725 [engine_partitioner.cc:1139][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.884.745 [engine_partitioner.cc:1142][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.884.910 [engine_partitioner.cc:1148][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [151] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.884.951 [engine_partitioner.cc:1155][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [24] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.884.994 [engine_partitioner.cc:1164][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [29] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.885.020 [graph_builder.cc:865][EVENT]167352 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [336] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:32.885.354 [logger.cc:1071] 167352 ModelBindStream: model_id=832, stream_id=1089, flag=0. [INFO] GE(164046,python):2024-01-10-11:37:32.885.398 [task_generator.cc:804][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [92] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.885.470 [task_generator.cc:805][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [58] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.886.276 [task_generator.cc:814][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [791] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.886.296 [task_generator.cc:954][EVENT]167352 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [991] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.886.358 [task_generator.cc:967][EVENT]167352 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [34] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:32.886.376 [logger.cc:1084] 167352 ModelUnbindStream: model_id=832, stream_id=1089, [INFO] GE(164046,python):2024-01-10-11:37:32.886.568 [graph_manager.cc:1152][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2050] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.886.588 [graph_manager.cc:1164][EVENT]167352 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:32.886.621 [graph_manager.cc:1271][EVENT]167352 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [44668] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.886.631 [graph_manager.cc:1272][EVENT]167352 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:37:32.886.944 [atrace_api.c:93](tid:167352) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:37:32.886.959 [atrace_api.c:95](tid:167352) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:37:32.893.759 [graph_converter.cc:838][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [2130] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.893.932 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [126] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.894.577 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [621] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.894.866 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [262] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.894.890 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [288] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.895.149 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [247] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.895.197 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [27] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.895.234 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.895.480 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [235] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.895.582 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [81] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.895.598 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [97] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.895.633 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [24] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.895.662 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.895.705 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.895.808 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [93] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.895.891 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [71] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.895.902 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [83] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.895.933 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.895.961 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.895.975 [graph_converter.cc:849][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [2174] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.896.271 [graph_converter.cc:853][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [286] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.897.225 [graph_converter.cc:857][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [937] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.897.419 [graph_converter.cc:862][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [167] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.970.009 [graph_var_manager.cc:1424][EVENT]167353 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:32.970.100 [graph_manager.cc:1248][EVENT]167353 PreRun:PreRun start: graph node size 4, session id 4, graph id 3, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:32.970.393 [atrace_api.c:28](tid:167353) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:32.970.423 [trace_rb_log.c:84](tid:167353) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:32.970.436 [atrace_api.c:32](tid:167353) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:32.970.454 [client_manager.cpp:157][SetProfilingCallback][tid:167353] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:32.970.834 [parallel_partitioner.cc:165][EVENT]167353 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.970.871 [parallel_partitioner.cc:178][EVENT]167353 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.970.916 [graph_prepare.cc:1378][EVENT]167353 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.971.078 [graph_manager.cc:1050][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [179] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.971.101 [graph_manager.cc:1052][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.971.236 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.971.301 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.971.349 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.971.361 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.971.408 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.971.422 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.971.438 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.971.535 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.971.554 [graph_manager.cc:1054][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [439] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.971.786 [graph_manager.cc:1055][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [218] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.972.912 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:32.972.942 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.972.953 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.972.963 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [417] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.972.972 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.972.981 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:32.972.990 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.972.998 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.973.007 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [7] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.976.624 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:32.976.656 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.976.668 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.976.677 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [372] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.976.689 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.976.709 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:32.976.719 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.976.732 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.976.741 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.978.060 [graph_manager.cc:1056][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [6254] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.978.131 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.978.150 [graph_prepare.cc:1982][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.978.692 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:32.978.718 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.978.730 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.978.739 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [293] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.978.748 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.978.757 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:32.978.766 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.978.775 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.978.783 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.978.830 [graph_prepare.cc:1983][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [666] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.978.855 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.978.866 [graph_prepare.cc:1984][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.978.880 [graph_prepare.cc:1985][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.978.894 [graph_prepare.cc:1986][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.978.906 [graph_prepare.cc:1987][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.978.932 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.978.944 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.978.957 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.979.048 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.979.059 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.979.068 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.979.077 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.979.086 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.979.095 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.979.103 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.979.111 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.979.120 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.979.128 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.979.137 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.979.145 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.979.153 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.979.162 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.979.170 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.979.178 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:32.979.201 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.979.214 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.979.247 [graph_prepare.cc:1988][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [320] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.979.263 [graph_manager.cc:1065][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1167] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.991.525 [graph_manager.cc:1077][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12241] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.991.636 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:32.991.688 [graph_manager.cc:1080][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [115] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.002.083 [graph_manager.cc:1081][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [10376] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.002.129 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.002.145 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.002.157 [graph_manager.cc:1082][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.002.187 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.002.202 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.002.216 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.002.386 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [159] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.002.404 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.002.509 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [93] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.002.525 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.002.575 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [39] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.002.596 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.002.616 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.002.748 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [121] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.002.767 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.002.780 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.002.791 [graph_manager.cc:2700][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [609] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.003.137 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [4] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.003.166 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.003.177 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.003.187 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.003.196 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.003.205 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CastRemovePass is [46] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.003.214 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [4] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.003.222 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.003.231 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [10] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.003.240 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [4] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.003.248 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [18] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.003.257 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [101] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.003.265 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [21] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.003.274 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [10] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.003.282 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [5] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.003.292 [graph_manager.cc:2741][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [483] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.003.302 [graph_manager.cc:2752][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.003.326 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.003.339 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.003.363 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.003.378 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.003.391 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.003.404 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.003.425 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.003.445 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.003.458 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.003.468 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.003.481 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.003.493 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.003.515 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.003.528 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.003.537 [graph_manager.cc:2810][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [215] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.003.583 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.003.595 [graph_manager.cc:2821][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [50] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.003.623 [graph_manager.cc:1087][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1447] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.004.210 [graph_manager.cc:1088][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [573] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.004.273 [graph_manager.cc:1089][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [32] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.004.297 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.004.315 [graph_manager.cc:1097][EVENT]167353 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:33.004.337 [graph_manager.cc:3325][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.013.967 [engine_place.cc:144][EVENT]167353 Run:The time cost of AIcoreEngine::CheckSupported is [9369] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.014.001 [engine_place.cc:144][EVENT]167353 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.014.012 [engine_place.cc:144][EVENT]167353 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.014.105 [graph_manager.cc:3351][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [9754] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.014.124 [graph_manager.cc:3364][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.014.200 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.014.243 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.014.416 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [162] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.014.461 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.014.506 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [34] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.014.541 [graph_manager.cc:3405][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [404] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.014.560 [graph_manager.cc:3412][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.101 [graph_manager.cc:3422][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [188525] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.155 [graph_manager.cc:3428][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.329 [graph_manager.cc:3467][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [152] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.349 [graph_manager.cc:3377][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [189214] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.366 [graph_manager.cc:1106][EVENT]167353 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [199036] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.379 [graph_manager.cc:1115][EVENT]167353 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:33.203.405 [graph_manager.cc:1130][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.438 [graph_manager.cc:1131][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.468 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.488 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.499 [graph_manager.cc:2837][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [44] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.644 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [30] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.203.658 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.203.669 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.203.697 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.203.707 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [12] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.203.716 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [15] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:33.203.727 [graph_manager.cc:2864][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [208] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.739 [graph_manager.cc:2872][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.760 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.775 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.792 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.806 [compile_nodes_pass.cc:88][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.816 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.826 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.936 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [101] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.985 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.203.999 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.204.014 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.204.029 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.204.038 [graph_manager.cc:2927][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [283] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.204.051 [graph_manager.cc:2937][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.204.066 [graph_manager.cc:2943][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.204.078 [graph_manager.cc:2950][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.204.301 [graph_manager.cc:2958][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [57] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.204.344 [graph_manager.cc:1132][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [892] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.204.425 [graph_manager.cc:1135][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [68] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.204.460 [graph_manager.cc:2975][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.204.491 [graph_manager.cc:2981][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.204.506 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.204.516 [graph_manager.cc:2986][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.204.525 [graph_manager.cc:1136][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [85] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.204.882 [graph_manager.cc:3555][EVENT]167353 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [317] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.205.020 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [34] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.205.052 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.205.200 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [137] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.205.238 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.205.282 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.205.308 [graph_builder.cc:865][EVENT]167353 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [354] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:33.205.755 [logger.cc:1071] 167353 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:37:33.205.790 [task_generator.cc:804][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [141] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.205.875 [task_generator.cc:805][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [73] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.207.768 [task_generator.cc:814][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1877] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.207.783 [task_generator.cc:954][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2134] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.207.854 [task_generator.cc:967][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [37] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:33.207.881 [logger.cc:1084] 167353 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:37:33.208.888 [graph_manager.cc:1152][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4330] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.208.923 [graph_manager.cc:1164][EVENT]167353 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:33.208.958 [graph_manager.cc:1271][EVENT]167353 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [238210] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.208.970 [graph_manager.cc:1272][EVENT]167353 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:37:33.209.288 [atrace_api.c:93](tid:167353) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:37:33.209.306 [atrace_api.c:95](tid:167353) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:37:33.234.834 [graph_converter.cc:838][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [9542] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.235.061 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [173] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.236.715 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [1625] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.237.124 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [379] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.237.152 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [409] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.237.423 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [258] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.237.513 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [65] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.237.587 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [54] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.238.078 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [474] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.238.315 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [205] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.238.341 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [232] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.238.413 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [59] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.238.476 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [50] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.238.541 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [51] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.238.775 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [220] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.238.984 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [190] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.239.003 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [212] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.239.071 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [58] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.239.133 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [49] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.239.152 [graph_converter.cc:849][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4269] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.239.933 [graph_converter.cc:853][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [755] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.242.066 [graph_converter.cc:857][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2107] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.242.479 [graph_converter.cc:862][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [382] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.319.820 [graph_var_manager.cc:1424][EVENT]167352 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:33.319.915 [graph_manager.cc:1248][EVENT]167352 PreRun:PreRun start: graph node size 3, session id 5, graph id 4, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:33.320.207 [atrace_api.c:28](tid:167352) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:33.320.242 [trace_rb_log.c:84](tid:167352) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:33.320.255 [atrace_api.c:32](tid:167352) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:33.320.276 [client_manager.cpp:157][SetProfilingCallback][tid:167352] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:33.320.668 [parallel_partitioner.cc:165][EVENT]167352 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.320.703 [parallel_partitioner.cc:178][EVENT]167352 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.320.747 [graph_prepare.cc:1378][EVENT]167352 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.320.882 [graph_manager.cc:1050][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [150] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.320.905 [graph_manager.cc:1052][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.321.025 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.321.053 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.321.100 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.321.113 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.321.158 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.321.170 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.321.187 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.321.287 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.321.332 [graph_manager.cc:1054][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [415] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.321.570 [graph_manager.cc:1055][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [225] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.322.526 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:33.322.555 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.322.567 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.322.576 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [256] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.322.585 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.322.594 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:33.322.603 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [87] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.322.612 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.322.620 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.324.572 [graph_manager.cc:1056][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2981] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.324.638 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.324.657 [graph_prepare.cc:1982][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [52] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.325.020 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:33.325.044 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.054 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.063 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [179] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.073 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.081 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:33.325.090 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.099 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.118 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.162 [graph_prepare.cc:1983][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [491] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.325.185 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.325.197 [graph_prepare.cc:1984][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.325.210 [graph_prepare.cc:1985][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.325.224 [graph_prepare.cc:1986][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.325.235 [graph_prepare.cc:1987][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.325.249 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.325.261 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.325.274 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.325.351 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.362 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.371 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.380 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.389 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.397 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.406 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.414 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.423 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.431 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.439 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.448 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.456 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.473 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.482 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.490 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.325.514 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.325.526 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.325.556 [graph_prepare.cc:1988][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [312] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.325.569 [graph_manager.cc:1065][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [963] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.337.603 [graph_manager.cc:1077][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12015] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.337.670 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.337.753 [graph_manager.cc:1080][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [114] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.176 [graph_manager.cc:1081][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3404] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.216 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.231 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.243 [graph_manager.cc:1082][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.273 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.286 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.300 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.334 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.349 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.363 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.376 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.424 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [27] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.443 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.461 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.484 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.499 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.510 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.519 [graph_manager.cc:2700][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [251] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.621 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.341.634 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.341.643 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.341.651 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.341.660 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.341.669 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CastRemovePass is [8] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.341.677 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [4] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.341.713 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.341.724 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.341.733 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.341.741 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.341.749 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.341.758 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.341.766 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.341.774 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.341.784 [graph_manager.cc:2741][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [247] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.800 [graph_manager.cc:2752][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.823 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.834 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.852 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.866 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.877 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.889 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.907 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.920 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.932 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.942 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.954 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.965 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.982 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.341.994 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.342.003 [graph_manager.cc:2810][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [184] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.342.029 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.342.041 [graph_manager.cc:2821][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [30] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.342.070 [graph_manager.cc:1087][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [808] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.342.205 [graph_manager.cc:1088][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [122] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.342.243 [graph_manager.cc:1089][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.342.258 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.342.281 [graph_manager.cc:1097][EVENT]167352 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:33.342.302 [graph_manager.cc:3325][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.342.655 [engine_place.cc:144][EVENT]167352 Run:The time cost of AIcoreEngine::CheckSupported is [258] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.342.681 [engine_place.cc:144][EVENT]167352 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.342.691 [engine_place.cc:144][EVENT]167352 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.342.762 [graph_manager.cc:3351][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [447] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.342.779 [graph_manager.cc:3364][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.342.841 [engine_partitioner.cc:1139][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.342.858 [engine_partitioner.cc:1142][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.342.984 [engine_partitioner.cc:1148][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [116] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.343.022 [engine_partitioner.cc:1155][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [26] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.343.065 [engine_partitioner.cc:1164][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.343.095 [graph_manager.cc:3405][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [304] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.343.112 [graph_manager.cc:3412][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.351.725 [graph_manager.cc:3422][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [8599] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.351.761 [graph_manager.cc:3428][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.351.875 [graph_manager.cc:3467][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [95] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.351.893 [graph_manager.cc:3377][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [9103] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.351.909 [graph_manager.cc:1106][EVENT]167352 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [9613] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.351.921 [graph_manager.cc:1115][EVENT]167352 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:33.351.943 [graph_manager.cc:1130][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.351.984 [graph_manager.cc:1131][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.007 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.022 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.032 [graph_manager.cc:2837][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [33] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.101 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.352.113 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.352.122 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondRemovePass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.352.131 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.352.139 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [4] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.352.148 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.352.157 [graph_manager.cc:2864][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [109] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.169 [graph_manager.cc:2872][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.186 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.199 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.214 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.227 [compile_nodes_pass.cc:88][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.237 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.247 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.314 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [58] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.339 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.352 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.374 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.387 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.395 [graph_manager.cc:2927][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [212] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.407 [graph_manager.cc:2937][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.421 [graph_manager.cc:2943][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.432 [graph_manager.cc:2950][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.610 [graph_manager.cc:2958][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [34] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.641 [graph_manager.cc:1132][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [644] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.707 [graph_manager.cc:1135][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [53] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.737 [graph_manager.cc:2975][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.768 [graph_manager.cc:2981][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.781 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.791 [graph_manager.cc:2986][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.800 [graph_manager.cc:1136][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [77] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.906 [graph_manager.cc:3555][EVENT]167352 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [76] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.352.986 [engine_partitioner.cc:1139][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.353.001 [engine_partitioner.cc:1142][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.353.090 [engine_partitioner.cc:1148][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [79] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.353.118 [engine_partitioner.cc:1155][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.353.155 [engine_partitioner.cc:1164][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [26] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.353.175 [graph_builder.cc:865][EVENT]167352 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [218] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:33.353.455 [logger.cc:1071] 167352 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:37:33.353.485 [task_generator.cc:804][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [75] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.353.539 [task_generator.cc:805][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [42] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.354.212 [task_generator.cc:814][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [660] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.354.228 [task_generator.cc:954][EVENT]167352 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [819] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.354.287 [task_generator.cc:967][EVENT]167352 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [33] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:33.354.304 [logger.cc:1084] 167352 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:37:33.354.462 [graph_manager.cc:1152][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [1640] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.354.480 [graph_manager.cc:1164][EVENT]167352 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:33.354.511 [graph_manager.cc:1271][EVENT]167352 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [33928] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.354.523 [graph_manager.cc:1272][EVENT]167352 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:37:33.354.825 [atrace_api.c:93](tid:167352) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:37:33.354.842 [atrace_api.c:95](tid:167352) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:37:33.359.430 [graph_converter.cc:838][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1292] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.359.588 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [111] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.360.035 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [422] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.360.221 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [161] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.360.241 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [183] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.360.449 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [196] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.360.487 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.360.516 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.360.697 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [169] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.360.777 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [62] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.360.791 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [77] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.360.831 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.360.855 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.360.881 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.360.950 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [60] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.361.013 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [52] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.361.024 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [64] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.361.049 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.361.071 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.361.084 [graph_converter.cc:849][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1614] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.361.287 [graph_converter.cc:853][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [194] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.361.925 [graph_converter.cc:857][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [625] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.362.058 [graph_converter.cc:862][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [106] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.442.695 [graph_var_manager.cc:1424][EVENT]167354 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:33.442.792 [graph_manager.cc:1248][EVENT]167354 PreRun:PreRun start: graph node size 3, session id 6, graph id 5, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:33.443.739 [atrace_api.c:28](tid:167354) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:33.443.813 [trace_rb_log.c:84](tid:167354) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:33.443.828 [atrace_api.c:32](tid:167354) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:33.443.847 [client_manager.cpp:157][SetProfilingCallback][tid:167354] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:33.444.767 [parallel_partitioner.cc:165][EVENT]167354 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.444.805 [parallel_partitioner.cc:178][EVENT]167354 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.444.856 [graph_prepare.cc:1378][EVENT]167354 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.445.544 [graph_manager.cc:1050][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [709] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.445.575 [graph_manager.cc:1052][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.445.756 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.445.788 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.445.835 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.445.849 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.445.896 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.445.909 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.445.926 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.446.027 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.446.048 [graph_manager.cc:1054][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [440] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.446.289 [graph_manager.cc:1055][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [228] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.447.257 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:33.447.286 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.447.297 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.447.306 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferShapePass is [329] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.447.316 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.447.325 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:33.447.334 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.447.343 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.447.352 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.449.408 [graph_manager.cc:1056][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3099] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.449.474 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.449.492 [graph_prepare.cc:1982][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [48] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.449.903 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:33.449.940 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.449.952 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.449.962 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferShapePass is [208] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.449.971 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.449.980 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:33.449.989 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.449.998 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.450.006 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.450.047 [graph_prepare.cc:1983][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [541] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.450.070 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.450.081 [graph_prepare.cc:1984][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.450.095 [graph_prepare.cc:1985][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.450.113 [graph_prepare.cc:1986][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.450.125 [graph_prepare.cc:1987][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.450.141 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.450.153 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.450.167 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.450.246 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.450.257 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.450.266 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.450.275 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.450.284 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.450.300 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.450.309 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.450.318 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.450.326 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.450.335 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.450.344 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.450.352 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.450.361 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.450.369 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.450.377 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.450.386 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:33.450.407 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.450.420 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.450.450 [graph_prepare.cc:1988][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [314] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.450.463 [graph_manager.cc:1065][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1020] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.462.393 [graph_manager.cc:1077][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11910] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.462.458 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.462.503 [graph_manager.cc:1080][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [74] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.468.050 [graph_manager.cc:1081][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [5531] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.468.093 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.468.109 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.468.121 [graph_manager.cc:1082][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.468.163 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.468.176 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.468.191 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.468.328 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [127] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.468.346 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.468.425 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [67] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.468.440 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.468.482 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [32] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.468.502 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.468.520 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.468.595 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [65] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.468.612 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.468.625 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.468.635 [graph_manager.cc:2700][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [478] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.468.830 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.468.845 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.468.854 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.468.864 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.468.873 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.468.881 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CastRemovePass is [31] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.468.890 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.468.899 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.468.908 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [8] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.468.926 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [4] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.468.936 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.468.945 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.468.954 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.468.962 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [6] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.468.970 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.468.980 [graph_manager.cc:2741][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [327] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.468.990 [graph_manager.cc:2752][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.013 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.024 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.045 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.060 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.071 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.084 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.103 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.117 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.130 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.141 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.152 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.164 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.185 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.199 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.215 [graph_manager.cc:2810][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [206] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.253 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.469.264 [graph_manager.cc:2821][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [40] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.290 [graph_manager.cc:1087][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1140] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.797 [graph_manager.cc:1088][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [492] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.856 [graph_manager.cc:1089][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [25] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.876 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.469.892 [graph_manager.cc:1097][EVENT]167354 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:33.469.914 [graph_manager.cc:3325][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.477.271 [engine_place.cc:144][EVENT]167354 Run:The time cost of AIcoreEngine::CheckSupported is [7167] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.477.304 [engine_place.cc:144][EVENT]167354 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.477.315 [engine_place.cc:144][EVENT]167354 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.477.398 [graph_manager.cc:3351][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [7470] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.477.418 [graph_manager.cc:3364][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.477.489 [engine_partitioner.cc:1139][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.477.516 [engine_partitioner.cc:1142][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.477.659 [engine_partitioner.cc:1148][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [132] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.477.717 [engine_partitioner.cc:1155][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [44] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.477.767 [engine_partitioner.cc:1164][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.477.803 [graph_manager.cc:3405][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [373] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.477.821 [graph_manager.cc:3412][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.127 [graph_manager.cc:3422][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [70282] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.173 [graph_manager.cc:3428][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.320 [graph_manager.cc:3467][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [127] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.339 [graph_manager.cc:3377][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [70910] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.356 [graph_manager.cc:1106][EVENT]167354 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [78449] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.369 [graph_manager.cc:1115][EVENT]167354 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:33.548.392 [graph_manager.cc:1130][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.424 [graph_manager.cc:1131][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.451 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.469 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.479 [graph_manager.cc:2837][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [39] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.590 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.548.603 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.548.613 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.548.622 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.548.631 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.548.640 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [13] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:37:33.548.650 [graph_manager.cc:2864][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [153] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.662 [graph_manager.cc:2872][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.681 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.695 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.724 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.737 [compile_nodes_pass.cc:88][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.747 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.757 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.848 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [81] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.882 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.895 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.909 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.922 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.931 [graph_manager.cc:2927][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [254] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.943 [graph_manager.cc:2937][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.957 [graph_manager.cc:2943][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.548.968 [graph_manager.cc:2950][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.549.170 [graph_manager.cc:2958][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [44] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.549.203 [graph_manager.cc:1132][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [765] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.549.277 [graph_manager.cc:1135][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [62] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.549.315 [graph_manager.cc:2975][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.549.346 [graph_manager.cc:2981][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.549.360 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.549.370 [graph_manager.cc:2986][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.549.379 [graph_manager.cc:1136][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [87] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.549.673 [graph_manager.cc:3555][EVENT]167354 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [249] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.549.861 [engine_partitioner.cc:1139][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.549.890 [engine_partitioner.cc:1142][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.550.006 [engine_partitioner.cc:1148][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [105] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.550.040 [engine_partitioner.cc:1155][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.550.079 [engine_partitioner.cc:1164][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.550.103 [graph_builder.cc:865][EVENT]167354 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [301] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:33.550.580 [logger.cc:1071] 167354 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:37:33.550.614 [task_generator.cc:804][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [171] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.550.691 [task_generator.cc:805][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [64] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.552.021 [task_generator.cc:814][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1314] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.552.037 [task_generator.cc:954][EVENT]167354 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1595] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.552.103 [task_generator.cc:967][EVENT]167354 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [35] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:33.552.125 [logger.cc:1084] 167354 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:37:33.553.794 [graph_manager.cc:1152][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4376] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.553.830 [graph_manager.cc:1164][EVENT]167354 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:33.553.866 [graph_manager.cc:1271][EVENT]167354 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [109185] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.553.878 [graph_manager.cc:1272][EVENT]167354 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:37:33.554.191 [atrace_api.c:93](tid:167354) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:37:33.554.208 [atrace_api.c:95](tid:167354) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:37:33.571.366 [graph_converter.cc:838][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [5992] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.571.568 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [148] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.572.645 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [1051] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.572.937 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [249] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.572.963 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [277] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.573.200 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [224] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.573.266 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [43] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.573.319 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.573.658 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [325] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.573.856 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [171] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.573.879 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [195] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.573.930 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [40] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.573.975 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [33] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.574.021 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.574.188 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [155] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.574.332 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [129] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.574.347 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [145] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.574.395 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.574.443 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.574.459 [graph_converter.cc:849][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [3047] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.574.974 [graph_converter.cc:853][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [504] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.576.366 [graph_converter.cc:857][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1368] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.576.652 [graph_converter.cc:862][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [255] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.655.046 [graph_var_manager.cc:1424][EVENT]167354 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:33.655.145 [graph_manager.cc:1248][EVENT]167354 PreRun:PreRun start: graph node size 4, session id 7, graph id 6, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:33.655.395 [atrace_api.c:28](tid:167354) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:33.655.422 [trace_rb_log.c:84](tid:167354) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:33.655.461 [atrace_api.c:32](tid:167354) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:33.655.479 [client_manager.cpp:157][SetProfilingCallback][tid:167354] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:33.655.869 [parallel_partitioner.cc:165][EVENT]167354 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.655.905 [parallel_partitioner.cc:178][EVENT]167354 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.655.951 [graph_prepare.cc:1378][EVENT]167354 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.656.125 [graph_manager.cc:1050][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [190] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.656.149 [graph_manager.cc:1052][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.656.284 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.656.314 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.656.362 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.656.375 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.656.420 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.656.434 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.656.452 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.656.552 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.656.573 [graph_manager.cc:1054][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [412] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.656.810 [graph_manager.cc:1055][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [224] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.658.002 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:33.658.034 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.658.045 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.658.054 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferShapePass is [482] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.658.063 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [15] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.658.083 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:33.658.093 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.658.102 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.658.110 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.661.016 [graph_manager.cc:1056][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [4185] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.661.086 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.661.105 [graph_prepare.cc:1982][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [55] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.661.658 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:33.661.710 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.661.724 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.661.734 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferShapePass is [348] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.661.744 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.661.753 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:33.661.762 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.661.770 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.661.778 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.661.805 [graph_prepare.cc:1983][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [686] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.661.829 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.661.840 [graph_prepare.cc:1984][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.661.854 [graph_prepare.cc:1985][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.661.868 [graph_prepare.cc:1986][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.661.878 [graph_prepare.cc:1987][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.661.892 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.661.915 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.661.930 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.662.022 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.662.035 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.662.045 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.662.054 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.662.062 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.662.071 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.662.079 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.662.088 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.662.096 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.662.105 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.662.113 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.662.121 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.662.130 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.662.138 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.662.146 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.662.155 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.662.177 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.662.189 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.662.222 [graph_prepare.cc:1988][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [335] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.662.234 [graph_manager.cc:1065][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1183] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.675.465 [graph_manager.cc:1077][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13212] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.675.573 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.675.625 [graph_manager.cc:1080][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [112] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.686.671 [graph_manager.cc:1081][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [11029] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.686.717 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.686.733 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.686.746 [graph_manager.cc:1082][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.686.776 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.686.789 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.686.803 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.686.956 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [142] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.686.974 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.069 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [84] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.084 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.134 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.155 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.175 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.263 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [78] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.282 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.295 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.305 [graph_manager.cc:2700][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [534] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.533 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of EnterPass is [4] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.687.560 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.687.570 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.687.580 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.687.589 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.687.598 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CastRemovePass is [40] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.687.606 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [5] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.687.615 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [5] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.687.624 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [10] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.687.633 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [4] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.687.641 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.687.650 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.687.658 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.687.667 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [7] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.687.675 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.687.685 [graph_manager.cc:2741][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [361] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.694 [graph_manager.cc:2752][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.720 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.732 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.754 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.770 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.782 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.795 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.816 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.837 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.851 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.861 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.874 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.885 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.907 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.921 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.930 [graph_manager.cc:2810][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [215] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.687.975 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of IdentityPass is [5] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.687.986 [graph_manager.cc:2821][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [46] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.688.013 [graph_manager.cc:1087][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1250] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.688.578 [graph_manager.cc:1088][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [550] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.688.640 [graph_manager.cc:1089][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.688.662 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.688.680 [graph_manager.cc:1097][EVENT]167354 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:33.688.703 [graph_manager.cc:3325][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.698.234 [engine_place.cc:144][EVENT]167354 Run:The time cost of AIcoreEngine::CheckSupported is [9306] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.698.268 [engine_place.cc:144][EVENT]167354 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.698.279 [engine_place.cc:144][EVENT]167354 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.698.371 [graph_manager.cc:3351][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [9654] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.698.390 [graph_manager.cc:3364][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.698.469 [engine_partitioner.cc:1139][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.698.511 [engine_partitioner.cc:1142][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.698.679 [engine_partitioner.cc:1148][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [157] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.698.720 [engine_partitioner.cc:1155][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [27] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.698.768 [engine_partitioner.cc:1164][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.698.801 [graph_manager.cc:3405][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [397] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.698.820 [graph_manager.cc:3412][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.822.572 [graph_manager.cc:3422][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [123738] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.822.622 [graph_manager.cc:3428][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.822.794 [graph_manager.cc:3467][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [150] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.822.814 [graph_manager.cc:3377][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [124412] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.822.832 [graph_manager.cc:1106][EVENT]167354 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [134136] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.822.847 [graph_manager.cc:1115][EVENT]167354 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:33.822.874 [graph_manager.cc:1130][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.822.908 [graph_manager.cc:1131][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.822.936 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.822.956 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.822.966 [graph_manager.cc:2837][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [43] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.101 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [27] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.823.115 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.823.124 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.823.149 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.823.159 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [7] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.823.168 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [17] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:33.823.178 [graph_manager.cc:2864][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [194] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.191 [graph_manager.cc:2872][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.210 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.225 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.242 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.256 [compile_nodes_pass.cc:88][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.265 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.276 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.384 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [97] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.435 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.449 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.463 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.477 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.487 [graph_manager.cc:2927][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [282] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.499 [graph_manager.cc:2937][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.514 [graph_manager.cc:2943][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.525 [graph_manager.cc:2950][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.749 [graph_manager.cc:2958][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [55] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.791 [graph_manager.cc:1132][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [871] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.873 [graph_manager.cc:1135][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [69] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.910 [graph_manager.cc:2975][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.945 [graph_manager.cc:2981][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.962 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.971 [graph_manager.cc:2986][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.823.980 [graph_manager.cc:1136][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [90] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.824.303 [graph_manager.cc:3555][EVENT]167354 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [284] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.824.438 [engine_partitioner.cc:1139][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [32] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.824.470 [engine_partitioner.cc:1142][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.824.614 [engine_partitioner.cc:1148][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [132] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.824.651 [engine_partitioner.cc:1155][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.824.694 [engine_partitioner.cc:1164][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.824.719 [graph_builder.cc:865][EVENT]167354 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [342] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:33.825.098 [logger.cc:1071] 167354 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:37:33.825.133 [task_generator.cc:804][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [89] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.825.215 [task_generator.cc:805][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [70] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.827.015 [task_generator.cc:814][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1785] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.827.034 [task_generator.cc:954][EVENT]167354 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1991] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.827.105 [task_generator.cc:967][EVENT]167354 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [38] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:33.827.133 [logger.cc:1084] 167354 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:37:33.828.365 [graph_manager.cc:1152][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4351] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.828.415 [graph_manager.cc:1164][EVENT]167354 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:33.828.453 [graph_manager.cc:1271][EVENT]167354 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [172671] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.828.466 [graph_manager.cc:1272][EVENT]167354 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:37:33.828.787 [atrace_api.c:93](tid:167354) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:37:33.828.808 [atrace_api.c:95](tid:167354) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:37:33.857.881 [graph_converter.cc:838][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [10936] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.858.103 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [166] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.859.592 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [1461] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.859.985 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [362] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.860.013 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [393] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.860.273 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [246] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.860.358 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [60] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.860.427 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [50] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.860.880 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [436] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.861.097 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [189] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.861.119 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [213] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.861.184 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [53] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.861.242 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [46] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.861.301 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [46] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.861.518 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [205] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.861.728 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [193] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.861.749 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [216] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.861.813 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [54] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.861.870 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [45] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.861.889 [graph_converter.cc:849][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [3958] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.862.610 [graph_converter.cc:853][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [695] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.864.538 [graph_converter.cc:857][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1903] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.864.920 [graph_converter.cc:862][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [351] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.937.049 [graph_var_manager.cc:1424][EVENT]167352 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:33.937.146 [graph_manager.cc:1248][EVENT]167352 PreRun:PreRun start: graph node size 4, session id 8, graph id 7, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:33.937.423 [atrace_api.c:28](tid:167352) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:33.937.455 [trace_rb_log.c:84](tid:167352) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:33.937.468 [atrace_api.c:32](tid:167352) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:33.937.486 [client_manager.cpp:157][SetProfilingCallback][tid:167352] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:33.937.921 [parallel_partitioner.cc:165][EVENT]167352 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.937.961 [parallel_partitioner.cc:178][EVENT]167352 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.938.007 [graph_prepare.cc:1378][EVENT]167352 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.938.138 [graph_manager.cc:1050][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [147] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.938.161 [graph_manager.cc:1052][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.938.294 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.938.324 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.938.372 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.938.385 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.938.430 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.938.443 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.938.460 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.938.555 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.938.602 [graph_manager.cc:1054][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [428] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.938.838 [graph_manager.cc:1055][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [223] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.939.902 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:33.939.930 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.939.941 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.939.951 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [323] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.939.960 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.939.969 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:33.939.978 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [81] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.939.987 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.939.995 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.942.111 [graph_manager.cc:1056][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3254] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.942.179 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.942.197 [graph_prepare.cc:1982][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [52] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.942.657 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:33.942.684 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.942.696 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.942.706 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [232] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.942.718 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.942.727 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:33.942.735 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.942.747 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.942.756 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.942.825 [graph_prepare.cc:1983][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [614] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.942.853 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.942.865 [graph_prepare.cc:1984][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.942.879 [graph_prepare.cc:1985][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.942.892 [graph_prepare.cc:1986][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.942.904 [graph_prepare.cc:1987][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.942.917 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.942.930 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.942.943 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.943.035 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.943.049 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.943.058 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.943.071 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.943.082 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.943.091 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.943.102 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.943.111 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.943.122 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.943.133 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.943.145 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.943.154 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.943.162 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.943.180 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.943.189 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.943.197 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.943.220 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.943.232 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.943.264 [graph_prepare.cc:1988][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [352] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.943.279 [graph_manager.cc:1065][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1135] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.955.610 [graph_manager.cc:1077][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12312] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.955.741 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.955.793 [graph_manager.cc:1080][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [146] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.962.531 [graph_manager.cc:1081][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [6722] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.962.575 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.962.590 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.962.601 [graph_manager.cc:1082][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.962.633 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.962.647 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.962.661 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.962.693 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.962.706 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.962.720 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.962.733 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.962.773 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [30] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.962.808 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.962.827 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.962.855 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.962.870 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.962.883 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.962.892 [graph_manager.cc:2700][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [265] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.009 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.963.022 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.963.032 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.963.041 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.963.050 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.963.059 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CastRemovePass is [8] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.963.068 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.963.076 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.963.085 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.963.093 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.963.102 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.963.110 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.963.119 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.963.127 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.963.135 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.963.145 [graph_manager.cc:2741][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [235] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.162 [graph_manager.cc:2752][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.184 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.195 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.212 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.227 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.239 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.251 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.270 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.283 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.296 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.306 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.318 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.330 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.347 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.360 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.369 [graph_manager.cc:2810][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [189] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.397 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:33.963.408 [graph_manager.cc:2821][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [30] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.435 [graph_manager.cc:1087][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [814] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.573 [graph_manager.cc:1088][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [124] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.613 [graph_manager.cc:1089][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.630 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.963.644 [graph_manager.cc:1097][EVENT]167352 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:33.963.674 [graph_manager.cc:3325][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.964.058 [engine_place.cc:144][EVENT]167352 Run:The time cost of AIcoreEngine::CheckSupported is [281] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.964.086 [engine_place.cc:144][EVENT]167352 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.964.097 [engine_place.cc:144][EVENT]167352 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.964.170 [graph_manager.cc:3351][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [482] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.964.187 [graph_manager.cc:3364][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.964.253 [engine_partitioner.cc:1139][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.964.270 [engine_partitioner.cc:1142][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.964.423 [engine_partitioner.cc:1148][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [142] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.964.464 [engine_partitioner.cc:1155][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [29] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.964.510 [engine_partitioner.cc:1164][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.964.541 [graph_manager.cc:3405][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [342] micro second. [INFO] GE(164046,python):2024-01-10-11:37:33.964.558 [graph_manager.cc:3412][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.055 [graph_manager.cc:3422][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [41483] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.101 [graph_manager.cc:3428][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.249 [graph_manager.cc:3467][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [127] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.267 [graph_manager.cc:3377][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [42069] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.283 [graph_manager.cc:1106][EVENT]167352 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [42616] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.296 [graph_manager.cc:1115][EVENT]167352 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:34.006.319 [graph_manager.cc:1130][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.362 [graph_manager.cc:1131][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.387 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.404 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.414 [graph_manager.cc:2837][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.498 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.006.511 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.006.520 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.006.529 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.006.538 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.006.546 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.006.556 [graph_manager.cc:2864][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [127] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.568 [graph_manager.cc:2872][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.586 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.599 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.614 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.628 [compile_nodes_pass.cc:88][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.637 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.647 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.726 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [69] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.757 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.769 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.790 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.804 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.814 [graph_manager.cc:2927][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [231] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.826 [graph_manager.cc:2937][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.840 [graph_manager.cc:2943][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.006.851 [graph_manager.cc:2950][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.007.050 [graph_manager.cc:2958][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [39] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.007.081 [graph_manager.cc:1132][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [705] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.007.156 [graph_manager.cc:1135][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [62] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.007.189 [graph_manager.cc:2975][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.007.220 [graph_manager.cc:2981][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.007.234 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.007.244 [graph_manager.cc:2986][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.007.253 [graph_manager.cc:1136][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [81] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.007.373 [graph_manager.cc:3555][EVENT]167352 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [87] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.007.467 [engine_partitioner.cc:1139][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.007.483 [engine_partitioner.cc:1142][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.007.604 [engine_partitioner.cc:1148][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [110] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.007.637 [engine_partitioner.cc:1155][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.007.676 [engine_partitioner.cc:1164][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [29] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.007.699 [graph_builder.cc:865][EVENT]167352 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [267] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:34.008.001 [logger.cc:1071] 167352 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:37:34.008.031 [task_generator.cc:804][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [80] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.008.091 [task_generator.cc:805][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [48] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.008.747 [task_generator.cc:814][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [642] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.008.761 [task_generator.cc:954][EVENT]167352 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [811] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.008.820 [task_generator.cc:967][EVENT]167352 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [33] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:34.008.836 [logger.cc:1084] 167352 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:37:34.009.366 [graph_manager.cc:1152][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2087] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.009.399 [graph_manager.cc:1164][EVENT]167352 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:34.009.433 [graph_manager.cc:1271][EVENT]167352 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [71599] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.009.445 [graph_manager.cc:1272][EVENT]167352 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:37:34.009.812 [atrace_api.c:93](tid:167352) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:37:34.009.833 [atrace_api.c:95](tid:167352) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:37:34.021.755 [graph_converter.cc:838][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [3719] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.021.922 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [119] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.022.404 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [461] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.022.616 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [184] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.022.637 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [208] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.022.852 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [203] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.022.895 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.022.925 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.023.119 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [183] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.023.204 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [66] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.023.218 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [81] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.023.247 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.023.283 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.023.311 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.023.386 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [65] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.023.454 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [57] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.023.465 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [69] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.023.491 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.023.514 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.023.527 [graph_converter.cc:849][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1730] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.023.749 [graph_converter.cc:853][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [212] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.024.470 [graph_converter.cc:857][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [706] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.024.615 [graph_converter.cc:862][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [118] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.091.074 [graph_var_manager.cc:1424][EVENT]167354 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:34.091.171 [graph_manager.cc:1248][EVENT]167354 PreRun:PreRun start: graph node size 3, session id 9, graph id 8, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:34.091.448 [atrace_api.c:28](tid:167354) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:34.091.474 [trace_rb_log.c:84](tid:167354) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:34.091.487 [atrace_api.c:32](tid:167354) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:34.091.505 [client_manager.cpp:157][SetProfilingCallback][tid:167354] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:34.091.864 [parallel_partitioner.cc:165][EVENT]167354 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.091.899 [parallel_partitioner.cc:178][EVENT]167354 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.091.943 [graph_prepare.cc:1378][EVENT]167354 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.092.091 [graph_manager.cc:1050][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [164] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.092.115 [graph_manager.cc:1052][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.092.267 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.092.296 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.092.343 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.092.356 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.092.401 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.092.415 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.092.431 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.092.529 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.092.549 [graph_manager.cc:1054][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [394] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.092.784 [graph_manager.cc:1055][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [220] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.093.627 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:34.093.655 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.093.667 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.093.676 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferShapePass is [252] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.093.716 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.093.728 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:34.093.737 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [10] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.093.746 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.093.755 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.095.713 [graph_manager.cc:1056][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2908] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.095.778 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondRemovePass is [6] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.095.798 [graph_prepare.cc:1982][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [50] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.096.140 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:34.096.177 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.189 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.199 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferShapePass is [175] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.208 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.217 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:34.096.226 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.234 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [8] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.243 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferValuePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.269 [graph_prepare.cc:1983][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [457] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.096.291 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.096.302 [graph_prepare.cc:1984][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.096.316 [graph_prepare.cc:1985][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.096.331 [graph_prepare.cc:1986][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.096.344 [graph_prepare.cc:1987][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.096.358 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.096.371 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.096.388 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.096.468 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.480 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.489 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.498 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.507 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.515 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.537 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.548 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.559 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.567 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.578 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.591 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.600 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.608 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.620 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.629 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.096.651 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.096.666 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.096.697 [graph_prepare.cc:1988][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [344] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.096.712 [graph_manager.cc:1065][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [965] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.108.711 [graph_manager.cc:1077][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11977] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.108.780 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.108.827 [graph_manager.cc:1080][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [77] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.302 [graph_manager.cc:1081][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3455] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.347 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.362 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.373 [graph_manager.cc:1082][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.404 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.430 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.446 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.474 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.490 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.504 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.519 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.558 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.577 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.594 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.617 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.631 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.643 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.652 [graph_manager.cc:2700][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [253] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.755 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.112.771 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.112.781 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.112.790 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.112.798 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.112.807 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CastRemovePass is [9] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.112.815 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.112.824 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.112.832 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.112.853 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.112.865 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.112.873 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.112.882 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.112.890 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.112.898 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.112.911 [graph_manager.cc:2741][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [240] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.921 [graph_manager.cc:2752][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.947 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.958 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.975 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.112.990 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.002 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.014 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.031 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.045 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.057 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.068 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.081 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.092 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.109 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.122 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.131 [graph_manager.cc:2810][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [188] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.165 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.113.180 [graph_manager.cc:2821][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [33] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.206 [graph_manager.cc:1087][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [814] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.345 [graph_manager.cc:1088][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [124] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.384 [graph_manager.cc:1089][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.401 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.415 [graph_manager.cc:1097][EVENT]167354 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:34.113.435 [graph_manager.cc:3325][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.856 [engine_place.cc:144][EVENT]167354 Run:The time cost of AIcoreEngine::CheckSupported is [325] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.887 [engine_place.cc:144][EVENT]167354 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.897 [engine_place.cc:144][EVENT]167354 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.967 [graph_manager.cc:3351][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [515] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.113.986 [graph_manager.cc:3364][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.114.054 [engine_partitioner.cc:1139][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.114.071 [engine_partitioner.cc:1142][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.114.197 [engine_partitioner.cc:1148][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [116] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.114.238 [engine_partitioner.cc:1155][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [27] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.114.281 [engine_partitioner.cc:1164][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [32] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.114.313 [graph_manager.cc:3405][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [314] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.114.330 [graph_manager.cc:3412][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.122.865 [graph_manager.cc:3422][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [8520] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.122.921 [graph_manager.cc:3428][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.055 [graph_manager.cc:3467][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [109] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.073 [graph_manager.cc:3377][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [9076] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.091 [graph_manager.cc:1106][EVENT]167354 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [9662] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.104 [graph_manager.cc:1115][EVENT]167354 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:34.123.126 [graph_manager.cc:1130][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.159 [graph_manager.cc:1131][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.183 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.200 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.210 [graph_manager.cc:2837][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.282 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.123.294 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.123.303 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondRemovePass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.123.312 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.123.321 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [4] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.123.330 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:34.123.340 [graph_manager.cc:2864][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [113] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.351 [graph_manager.cc:2872][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.369 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.382 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.397 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.421 [compile_nodes_pass.cc:88][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.431 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.441 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.514 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [64] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.543 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.556 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.569 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.581 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.591 [graph_manager.cc:2927][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [224] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.603 [graph_manager.cc:2937][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.617 [graph_manager.cc:2943][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.629 [graph_manager.cc:2950][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.810 [graph_manager.cc:2958][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.842 [graph_manager.cc:1132][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [669] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.909 [graph_manager.cc:1135][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [54] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.941 [graph_manager.cc:2975][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.973 [graph_manager.cc:2981][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.986 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.123.996 [graph_manager.cc:2986][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.124.006 [graph_manager.cc:1136][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [80] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.124.118 [graph_manager.cc:3555][EVENT]167354 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [80] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.124.214 [engine_partitioner.cc:1139][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.124.229 [engine_partitioner.cc:1142][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.124.321 [engine_partitioner.cc:1148][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [82] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.124.350 [engine_partitioner.cc:1155][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.124.386 [engine_partitioner.cc:1164][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [26] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.124.406 [graph_builder.cc:865][EVENT]167354 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [224] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:34.124.684 [logger.cc:1071] 167354 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:37:34.124.714 [task_generator.cc:804][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [77] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.124.770 [task_generator.cc:805][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [45] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.125.384 [task_generator.cc:814][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [599] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.125.398 [task_generator.cc:954][EVENT]167354 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [762] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.125.454 [task_generator.cc:967][EVENT]167354 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [32] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:34.125.471 [logger.cc:1084] 167354 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:37:34.125.630 [graph_manager.cc:1152][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [1601] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.125.648 [graph_manager.cc:1164][EVENT]167354 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:34.125.680 [graph_manager.cc:1271][EVENT]167354 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [33900] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.125.740 [graph_manager.cc:1272][EVENT]167354 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:37:34.126.051 [atrace_api.c:93](tid:167354) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:37:34.126.068 [atrace_api.c:95](tid:167354) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:37:34.130.599 [graph_converter.cc:838][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1273] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.130.756 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [113] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.131.201 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [421] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.131.389 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [163] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.131.420 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [196] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.131.635 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [203] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.131.675 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.131.705 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.131.890 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [174] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.131.972 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [63] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.131.986 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [77] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.132.015 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.132.039 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.132.064 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.132.133 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [59] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.132.196 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [53] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.132.207 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [64] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.132.232 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.132.254 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.132.267 [graph_converter.cc:849][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1628] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.132.474 [graph_converter.cc:853][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [198] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.133.120 [graph_converter.cc:857][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [630] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.133.252 [graph_converter.cc:862][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [107] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.197.633 [graph_var_manager.cc:1424][EVENT]167353 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:34.197.776 [graph_manager.cc:1248][EVENT]167353 PreRun:PreRun start: graph node size 4, session id 10, graph id 9, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:34.198.051 [atrace_api.c:28](tid:167353) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:34.198.078 [trace_rb_log.c:84](tid:167353) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:34.198.117 [atrace_api.c:32](tid:167353) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:34.198.136 [client_manager.cpp:157][SetProfilingCallback][tid:167353] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:34.198.538 [parallel_partitioner.cc:165][EVENT]167353 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.198.576 [parallel_partitioner.cc:178][EVENT]167353 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.198.620 [graph_prepare.cc:1378][EVENT]167353 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.198.799 [graph_manager.cc:1050][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [195] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.198.824 [graph_manager.cc:1052][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.198.962 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.198.991 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.199.038 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.199.050 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.199.097 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.199.111 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.199.128 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.199.226 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.199.247 [graph_manager.cc:1054][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [410] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.199.482 [graph_manager.cc:1055][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [220] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.200.623 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:34.200.652 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.200.663 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.200.673 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [446] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.200.683 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.200.692 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:34.200.711 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [15] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.200.721 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.200.730 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [7] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.203.748 [graph_manager.cc:1056][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [4248] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.203.820 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.203.839 [graph_prepare.cc:1982][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [55] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.204.401 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:34.204.429 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.440 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.450 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [352] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.459 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.469 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:34.204.477 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.486 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.495 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.521 [graph_prepare.cc:1983][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [668] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.204.544 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.204.556 [graph_prepare.cc:1984][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.204.570 [graph_prepare.cc:1985][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.204.584 [graph_prepare.cc:1986][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.204.595 [graph_prepare.cc:1987][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.204.610 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.204.632 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.204.647 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.204.737 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.749 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.758 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.767 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.775 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.784 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.792 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.801 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.809 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.817 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.826 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.834 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.843 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.852 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [7] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.860 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.869 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.204.891 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.204.903 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.204.935 [graph_prepare.cc:1988][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [331] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.204.948 [graph_manager.cc:1065][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1164] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.218.164 [graph_manager.cc:1077][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13196] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.218.249 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.218.299 [graph_manager.cc:1080][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [85] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.229.302 [graph_manager.cc:1081][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [10986] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.229.347 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.229.363 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.229.375 [graph_manager.cc:1082][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.229.405 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.229.420 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.229.434 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.229.584 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [139] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.229.602 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.229.705 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [93] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.229.723 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.229.773 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.229.794 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.229.814 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.229.901 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [77] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.229.919 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.229.932 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.229.942 [graph_manager.cc:2700][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [543] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.230.172 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.230.188 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AddNPass is [0] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.230.208 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.230.219 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [4] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.230.228 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.230.237 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CastRemovePass is [41] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.230.245 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.230.254 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.230.263 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [11] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.230.271 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.230.279 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.230.288 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.230.296 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [21] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.230.305 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [8] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.230.314 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.230.324 [graph_manager.cc:2741][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [363] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.230.333 [graph_manager.cc:2752][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.230.356 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.230.368 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.230.389 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.230.404 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.230.416 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.230.428 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.230.448 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.230.462 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.230.479 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.230.490 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.230.502 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.230.514 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.230.537 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.230.550 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.230.559 [graph_manager.cc:2810][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [208] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.230.603 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.230.615 [graph_manager.cc:2821][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [47] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.230.644 [graph_manager.cc:1087][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1250] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.231.216 [graph_manager.cc:1088][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [559] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.231.279 [graph_manager.cc:1089][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.231.301 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.231.322 [graph_manager.cc:1097][EVENT]167353 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:34.231.344 [graph_manager.cc:3325][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.241.739 [engine_place.cc:144][EVENT]167353 Run:The time cost of AIcoreEngine::CheckSupported is [10147] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.241.773 [engine_place.cc:144][EVENT]167353 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.241.784 [engine_place.cc:144][EVENT]167353 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.241.878 [graph_manager.cc:3351][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10520] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.241.897 [graph_manager.cc:3364][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.241.976 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [29] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.242.025 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.242.198 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [159] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.242.241 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [26] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.242.288 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.242.321 [graph_manager.cc:3405][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [410] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.242.339 [graph_manager.cc:3412][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.357.304 [graph_manager.cc:3422][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [114948] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.357.351 [graph_manager.cc:3428][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.357.521 [graph_manager.cc:3467][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [147] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.357.541 [graph_manager.cc:3377][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [115631] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.357.558 [graph_manager.cc:1106][EVENT]167353 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [126221] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.357.570 [graph_manager.cc:1115][EVENT]167353 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:34.357.596 [graph_manager.cc:1130][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.357.628 [graph_manager.cc:1131][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.357.658 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.357.677 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.357.743 [graph_manager.cc:2837][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [97] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.357.886 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [26] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.357.902 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.357.911 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.357.920 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.357.943 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.357.953 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [15] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:34.357.964 [graph_manager.cc:2864][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [195] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.357.976 [graph_manager.cc:2872][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.357.996 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.014 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.031 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.045 [compile_nodes_pass.cc:88][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.055 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.065 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.175 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [100] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.226 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.240 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.255 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.268 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.278 [graph_manager.cc:2927][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [285] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.290 [graph_manager.cc:2937][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.305 [graph_manager.cc:2943][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.316 [graph_manager.cc:2950][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.535 [graph_manager.cc:2958][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [56] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.571 [graph_manager.cc:1132][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [929] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.666 [graph_manager.cc:1135][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [66] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.705 [graph_manager.cc:2975][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.736 [graph_manager.cc:2981][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.751 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.761 [graph_manager.cc:2986][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.358.772 [graph_manager.cc:1136][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [87] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.359.096 [graph_manager.cc:3555][EVENT]167353 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [282] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.359.226 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [32] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.359.258 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.359.405 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [133] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.359.440 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.359.481 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [29] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.359.509 [graph_builder.cc:865][EVENT]167353 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [344] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:34.359.884 [logger.cc:1071] 167353 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:37:34.359.917 [task_generator.cc:804][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [89] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.359.997 [task_generator.cc:805][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [68] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.361.650 [task_generator.cc:814][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1636] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.361.667 [task_generator.cc:954][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1840] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.361.752 [task_generator.cc:967][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [38] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:34.361.780 [logger.cc:1084] 167353 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:37:34.363.118 [graph_manager.cc:1152][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4311] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.363.170 [graph_manager.cc:1164][EVENT]167353 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:34.363.208 [graph_manager.cc:1271][EVENT]167353 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [164761] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.363.220 [graph_manager.cc:1272][EVENT]167353 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:37:34.363.541 [atrace_api.c:93](tid:167353) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:37:34.363.561 [atrace_api.c:95](tid:167353) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:37:34.392.367 [graph_converter.cc:838][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [10810] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.392.593 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [168] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.394.144 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [1523] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.394.539 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [361] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.394.567 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [392] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.394.832 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [251] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.394.918 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [61] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.394.986 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [50] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.395.445 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [443] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.395.663 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [190] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.395.685 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [213] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.395.752 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [55] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.395.809 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [45] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.395.867 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [46] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.396.086 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [206] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.396.279 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [176] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.396.297 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [194] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.396.358 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [51] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.396.415 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [45] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.396.433 [graph_converter.cc:849][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4014] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.397.156 [graph_converter.cc:853][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [699] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.399.068 [graph_converter.cc:857][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1885] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.399.457 [graph_converter.cc:862][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [356] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.470.866 [graph_var_manager.cc:1424][EVENT]167352 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:34.470.963 [graph_manager.cc:1248][EVENT]167352 PreRun:PreRun start: graph node size 4, session id 11, graph id 10, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:34.471.245 [atrace_api.c:28](tid:167352) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:34.471.272 [trace_rb_log.c:84](tid:167352) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:34.471.285 [atrace_api.c:32](tid:167352) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:34.471.302 [client_manager.cpp:157][SetProfilingCallback][tid:167352] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:34.471.673 [parallel_partitioner.cc:165][EVENT]167352 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.471.710 [parallel_partitioner.cc:178][EVENT]167352 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.471.754 [graph_prepare.cc:1378][EVENT]167352 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.471.890 [graph_manager.cc:1050][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [152] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.471.911 [graph_manager.cc:1052][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.472.045 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.472.074 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.472.123 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.472.136 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.472.182 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.472.195 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.472.213 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.472.314 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.472.334 [graph_manager.cc:1054][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [410] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.472.594 [graph_manager.cc:1055][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [221] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.473.775 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:34.473.805 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.473.816 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.473.825 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [381] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.473.835 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.473.844 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:34.473.853 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [88] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.473.862 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.473.871 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.011 [graph_manager.cc:1056][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3397] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.476.080 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.099 [graph_prepare.cc:1982][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [54] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.476.536 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:34.476.561 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.572 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.583 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [250] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.594 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.605 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:34.476.614 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.625 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.636 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.674 [graph_prepare.cc:1983][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [562] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.476.699 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.476.711 [graph_prepare.cc:1984][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.476.724 [graph_prepare.cc:1985][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.476.741 [graph_prepare.cc:1986][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.476.754 [graph_prepare.cc:1987][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.476.770 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.476.781 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.476.794 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.476.885 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.899 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.908 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.917 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.926 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.935 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.944 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.952 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.961 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.969 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.978 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.986 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SnapshotPass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.476.994 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.477.003 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.477.020 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.477.028 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.477.054 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.477.067 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.477.103 [graph_prepare.cc:1988][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [338] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.477.115 [graph_manager.cc:1065][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1072] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.489.287 [graph_manager.cc:1077][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12151] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.489.359 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.489.413 [graph_manager.cc:1080][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [89] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.053 [graph_manager.cc:1081][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [4622] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.095 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.110 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.122 [graph_manager.cc:1082][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [34] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.153 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.167 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.181 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.214 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.227 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.242 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.255 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.295 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [30] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.323 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.341 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.371 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.385 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.396 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.405 [graph_manager.cc:2700][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [258] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.520 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.494.534 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AddNPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.494.543 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.494.552 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.494.561 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.494.569 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.494.578 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.494.587 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.494.596 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.494.604 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.494.612 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.494.621 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.494.629 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.494.637 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.494.645 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.494.655 [graph_manager.cc:2741][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [232] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.664 [graph_manager.cc:2752][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.693 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.704 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.721 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.735 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.746 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.758 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.777 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.790 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.803 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.813 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.826 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.838 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.858 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.871 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.880 [graph_manager.cc:2810][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [192] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.911 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.494.922 [graph_manager.cc:2821][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [33] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.494.949 [graph_manager.cc:1087][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [808] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.495.085 [graph_manager.cc:1088][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [124] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.495.126 [graph_manager.cc:1089][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.495.142 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.495.158 [graph_manager.cc:1097][EVENT]167352 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:34.495.187 [graph_manager.cc:3325][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.496.225 [engine_place.cc:144][EVENT]167352 Run:The time cost of AIcoreEngine::CheckSupported is [927] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.496.254 [engine_place.cc:144][EVENT]167352 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.496.264 [engine_place.cc:144][EVENT]167352 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.496.341 [graph_manager.cc:3351][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [1139] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.496.357 [graph_manager.cc:3364][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.496.416 [engine_partitioner.cc:1139][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.496.433 [engine_partitioner.cc:1142][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.496.603 [engine_partitioner.cc:1148][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [160] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.496.647 [engine_partitioner.cc:1155][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [30] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.496.694 [engine_partitioner.cc:1164][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.496.728 [graph_manager.cc:3405][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [358] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.496.746 [graph_manager.cc:3412][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.517.277 [graph_manager.cc:3422][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [20517] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.517.324 [graph_manager.cc:3428][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.517.471 [graph_manager.cc:3467][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [126] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.517.488 [graph_manager.cc:3377][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [21118] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.517.504 [graph_manager.cc:1106][EVENT]167352 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [22323] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.517.516 [graph_manager.cc:1115][EVENT]167352 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:34.517.540 [graph_manager.cc:1130][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.517.571 [graph_manager.cc:1131][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.517.618 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.517.636 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.517.646 [graph_manager.cc:2837][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.517.752 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.517.767 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.517.776 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.517.785 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.517.794 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [7] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.517.803 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.517.813 [graph_manager.cc:2864][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [150] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.517.824 [graph_manager.cc:2872][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.517.845 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.517.858 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.517.873 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.517.886 [compile_nodes_pass.cc:88][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.517.896 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.517.906 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.517.991 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [76] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.019 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.032 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.046 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.067 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.077 [graph_manager.cc:2927][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [236] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.090 [graph_manager.cc:2937][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.104 [graph_manager.cc:2943][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.115 [graph_manager.cc:2950][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.312 [graph_manager.cc:2958][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.344 [graph_manager.cc:1132][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [736] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.420 [graph_manager.cc:1135][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [63] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.451 [graph_manager.cc:2975][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.482 [graph_manager.cc:2981][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.496 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.506 [graph_manager.cc:2986][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.515 [graph_manager.cc:1136][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [80] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.633 [graph_manager.cc:3555][EVENT]167352 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [86] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.723 [engine_partitioner.cc:1139][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.739 [engine_partitioner.cc:1142][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.879 [engine_partitioner.cc:1148][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [130] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.913 [engine_partitioner.cc:1155][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.953 [engine_partitioner.cc:1164][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [29] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.518.975 [graph_builder.cc:865][EVENT]167352 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [288] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:34.519.286 [logger.cc:1071] 167352 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:37:34.519.316 [task_generator.cc:804][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [84] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.519.378 [task_generator.cc:805][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [48] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.519.987 [task_generator.cc:814][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [596] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.520.002 [task_generator.cc:954][EVENT]167352 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [771] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.520.060 [task_generator.cc:967][EVENT]167352 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [32] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:34.520.078 [logger.cc:1084] 167352 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:37:34.521.364 [graph_manager.cc:1152][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2821] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.521.399 [graph_manager.cc:1164][EVENT]167352 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:34.521.434 [graph_manager.cc:1271][EVENT]167352 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [49847] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.521.447 [graph_manager.cc:1272][EVENT]167352 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:37:34.521.772 [atrace_api.c:93](tid:167352) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:37:34.521.795 [atrace_api.c:95](tid:167352) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:37:34.545.351 [graph_converter.cc:838][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [7449] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.545.564 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [156] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.546.158 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [571] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.546.380 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [193] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.546.402 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [217] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.546.700 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [286] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.546.744 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [24] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.546.777 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.546.984 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [195] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.547.073 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [69] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.547.087 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [84] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.547.119 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.547.157 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.547.185 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.547.264 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [70] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.547.335 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [59] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.547.346 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [72] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.547.374 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.547.399 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.547.412 [graph_converter.cc:849][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [2010] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.547.652 [graph_converter.cc:853][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [230] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.548.413 [graph_converter.cc:857][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [744] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.548.565 [graph_converter.cc:862][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [125] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.612.827 [graph_var_manager.cc:1424][EVENT]167354 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:34.612.928 [graph_manager.cc:1248][EVENT]167354 PreRun:PreRun start: graph node size 4, session id 12, graph id 11, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:34.613.174 [atrace_api.c:28](tid:167354) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:34.613.203 [trace_rb_log.c:84](tid:167354) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:34.613.216 [atrace_api.c:32](tid:167354) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:34.613.234 [client_manager.cpp:157][SetProfilingCallback][tid:167354] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:34.613.631 [parallel_partitioner.cc:165][EVENT]167354 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.613.668 [parallel_partitioner.cc:178][EVENT]167354 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.613.747 [graph_prepare.cc:1378][EVENT]167354 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.613.881 [graph_manager.cc:1050][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [150] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.613.903 [graph_manager.cc:1052][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.614.037 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.614.098 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.614.146 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.614.159 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.614.205 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.614.219 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.614.235 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.614.336 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.614.355 [graph_manager.cc:1054][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [439] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.614.593 [graph_manager.cc:1055][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [225] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.615.635 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:34.615.664 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.615.676 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.615.686 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferShapePass is [340] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.615.695 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.615.704 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:34.615.713 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [25] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.615.722 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.615.730 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferValuePass is [10] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.617.844 [graph_manager.cc:1056][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3231] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.617.914 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.617.933 [graph_prepare.cc:1982][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [54] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.618.387 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:34.618.412 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.435 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.445 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferShapePass is [259] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.455 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.463 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:34.618.472 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.481 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.489 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.515 [graph_prepare.cc:1983][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [568] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.618.539 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.618.550 [graph_prepare.cc:1984][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.618.564 [graph_prepare.cc:1985][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.618.578 [graph_prepare.cc:1986][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.618.589 [graph_prepare.cc:1987][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.618.604 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.618.616 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.618.629 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.618.719 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.731 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.740 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.749 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.757 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.766 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.783 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.793 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.802 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of StopGradientPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.810 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.818 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.827 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.835 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.844 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.852 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.861 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.618.883 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.618.896 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.618.927 [graph_prepare.cc:1988][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [329] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.618.940 [graph_manager.cc:1065][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1061] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.631.354 [graph_manager.cc:1077][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12395] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.631.423 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.631.472 [graph_manager.cc:1080][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [82] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.150 [graph_manager.cc:1081][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [8659] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.194 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.211 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.223 [graph_manager.cc:1082][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.254 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.282 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.297 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.391 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [83] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.406 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.454 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.470 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.509 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [30] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.528 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.546 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.574 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.589 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.601 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.610 [graph_manager.cc:2700][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [361] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.731 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.640.745 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.640.755 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.640.764 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.640.773 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.640.782 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CastRemovePass is [9] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.640.790 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.640.799 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.640.808 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.640.817 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.640.835 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.640.844 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.640.853 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.640.861 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.640.870 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.640.880 [graph_manager.cc:2741][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [251] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.889 [graph_manager.cc:2752][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.911 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.923 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.941 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.956 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.968 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.980 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.640.998 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.641.012 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.641.025 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.641.037 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.641.050 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.641.061 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.641.080 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.641.094 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.641.103 [graph_manager.cc:2810][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [197] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.641.142 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.641.156 [graph_manager.cc:2821][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [34] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.641.184 [graph_manager.cc:1087][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [942] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.641.322 [graph_manager.cc:1088][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [125] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.641.362 [graph_manager.cc:1089][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.641.379 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.641.394 [graph_manager.cc:1097][EVENT]167354 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:34.641.413 [graph_manager.cc:3325][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.641.838 [engine_place.cc:144][EVENT]167354 Run:The time cost of AIcoreEngine::CheckSupported is [319] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.641.867 [engine_place.cc:144][EVENT]167354 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.641.876 [engine_place.cc:144][EVENT]167354 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.641.951 [graph_manager.cc:3351][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [524] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.641.968 [graph_manager.cc:3364][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.642.034 [engine_partitioner.cc:1139][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.642.051 [engine_partitioner.cc:1142][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.642.204 [engine_partitioner.cc:1148][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [143] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.642.247 [engine_partitioner.cc:1155][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [29] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.642.293 [engine_partitioner.cc:1164][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.642.325 [graph_manager.cc:3405][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [344] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.642.342 [graph_manager.cc:3412][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.660.500 [graph_manager.cc:3422][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [18145] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.660.558 [graph_manager.cc:3428][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.660.696 [graph_manager.cc:3467][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [117] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.660.713 [graph_manager.cc:3377][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [18734] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.660.730 [graph_manager.cc:1106][EVENT]167354 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [19322] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.660.742 [graph_manager.cc:1115][EVENT]167354 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:34.660.765 [graph_manager.cc:1130][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.660.797 [graph_manager.cc:1131][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.660.820 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.660.837 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.660.847 [graph_manager.cc:2837][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.660.930 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.660.942 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.660.952 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.660.960 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of BitcastPass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.660.969 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.660.978 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [8] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.660.987 [graph_manager.cc:2864][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [124] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.660.999 [graph_manager.cc:2872][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.018 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.031 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.046 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.069 [compile_nodes_pass.cc:88][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.079 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.089 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.168 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [69] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.197 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.210 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.223 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.236 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.245 [graph_manager.cc:2927][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [230] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.257 [graph_manager.cc:2937][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.271 [graph_manager.cc:2943][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.283 [graph_manager.cc:2950][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.467 [graph_manager.cc:2958][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.497 [graph_manager.cc:1132][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [689] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.568 [graph_manager.cc:1135][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [58] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.601 [graph_manager.cc:2975][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.632 [graph_manager.cc:2981][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.646 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.656 [graph_manager.cc:2986][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.665 [graph_manager.cc:1136][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [80] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.857 [graph_manager.cc:3555][EVENT]167354 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [99] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.957 [engine_partitioner.cc:1139][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.661.974 [engine_partitioner.cc:1142][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.662.094 [engine_partitioner.cc:1148][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [110] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.662.127 [engine_partitioner.cc:1155][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.662.165 [engine_partitioner.cc:1164][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [26] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.662.187 [graph_builder.cc:865][EVENT]167354 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [263] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:34.662.476 [logger.cc:1071] 167354 ModelBindStream: model_id=1856, stream_id=65, flag=0. [INFO] GE(164046,python):2024-01-10-11:37:34.662.508 [task_generator.cc:804][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [82] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.662.566 [task_generator.cc:805][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [46] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.663.326 [task_generator.cc:814][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [746] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.663.340 [task_generator.cc:954][EVENT]167354 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [915] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.663.397 [task_generator.cc:967][EVENT]167354 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [31] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:34.663.414 [logger.cc:1084] 167354 ModelUnbindStream: model_id=1856, stream_id=65, [INFO] GE(164046,python):2024-01-10-11:37:34.663.982 [graph_manager.cc:1152][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2230] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.664.015 [graph_manager.cc:1164][EVENT]167354 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:34.664.052 [graph_manager.cc:1271][EVENT]167354 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [50507] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.664.064 [graph_manager.cc:1272][EVENT]167354 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:37:34.664.377 [atrace_api.c:93](tid:167354) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:37:34.664.396 [atrace_api.c:95](tid:167354) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:37:34.675.553 [graph_converter.cc:838][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [3622] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.675.716 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [117] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.676.205 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [466] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.676.414 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [185] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.676.447 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [219] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.676.670 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [211] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.676.710 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.676.741 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.676.940 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [187] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.677.025 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [66] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.677.040 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [83] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.677.070 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.677.094 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.677.120 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.677.196 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [66] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.677.262 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [56] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.677.273 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [67] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.677.299 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.677.322 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.677.335 [graph_converter.cc:849][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1742] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.677.559 [graph_converter.cc:853][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [214] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.678.293 [graph_converter.cc:857][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [718] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.678.438 [graph_converter.cc:862][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [117] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.745.164 [graph_var_manager.cc:1424][EVENT]167353 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:34.745.262 [graph_manager.cc:1248][EVENT]167353 PreRun:PreRun start: graph node size 4, session id 13, graph id 12, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:34.745.502 [atrace_api.c:28](tid:167353) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:34.745.532 [trace_rb_log.c:84](tid:167353) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:34.745.546 [atrace_api.c:32](tid:167353) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:34.745.589 [client_manager.cpp:157][SetProfilingCallback][tid:167353] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:34.746.054 [parallel_partitioner.cc:165][EVENT]167353 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.746.094 [parallel_partitioner.cc:178][EVENT]167353 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.746.141 [graph_prepare.cc:1378][EVENT]167353 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.746.316 [graph_manager.cc:1050][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [193] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.746.339 [graph_manager.cc:1052][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.746.487 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.746.517 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.746.564 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.746.577 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.746.623 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.746.637 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.746.655 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.746.755 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.746.775 [graph_manager.cc:1054][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [424] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.747.013 [graph_manager.cc:1055][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [224] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.748.134 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:34.748.163 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.748.175 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.748.185 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [360] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.748.195 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.748.204 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:34.748.224 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [15] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.748.234 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.748.242 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.750.423 [graph_manager.cc:1056][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3390] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.750.492 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.750.511 [graph_prepare.cc:1982][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.750.954 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:34.750.980 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.750.992 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.002 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [246] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.011 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.020 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [4] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:34.751.029 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.038 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.046 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.073 [graph_prepare.cc:1983][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [548] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.751.097 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.751.108 [graph_prepare.cc:1984][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.751.122 [graph_prepare.cc:1985][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.751.137 [graph_prepare.cc:1986][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.751.148 [graph_prepare.cc:1987][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.751.163 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.751.175 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.751.200 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.751.292 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.303 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.313 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.321 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.330 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.338 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.347 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.355 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.364 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.372 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.381 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.389 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.397 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.405 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.413 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.422 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.751.443 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.751.457 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.751.489 [graph_prepare.cc:1988][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [331] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.751.501 [graph_manager.cc:1065][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1044] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.764.443 [graph_manager.cc:1077][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12922] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.764.506 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.764.570 [graph_manager.cc:1080][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [90] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.062 [graph_manager.cc:1081][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3474] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.103 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.118 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.130 [graph_manager.cc:1082][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [34] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.159 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.172 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.186 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.289 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [93] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.305 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.358 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [42] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.373 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.413 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [29] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.434 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.462 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.489 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.505 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.517 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.526 [graph_manager.cc:2700][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [371] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.642 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.768.658 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AddNPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.768.678 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.768.688 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.768.696 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.768.705 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.768.714 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.768.725 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.768.733 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.768.742 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.768.753 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.768.762 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.768.770 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.768.779 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.768.787 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.768.797 [graph_manager.cc:2741][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [253] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.806 [graph_manager.cc:2752][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.828 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.839 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.856 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.871 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.883 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.896 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.914 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.928 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.960 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.971 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.984 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.768.995 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.769.013 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.769.026 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.769.035 [graph_manager.cc:2810][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [211] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.769.064 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.769.074 [graph_manager.cc:2821][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.769.101 [graph_manager.cc:1087][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [953] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.769.238 [graph_manager.cc:1088][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [124] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.769.279 [graph_manager.cc:1089][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.769.296 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.769.312 [graph_manager.cc:1097][EVENT]167353 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:34.769.332 [graph_manager.cc:3325][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.769.733 [engine_place.cc:144][EVENT]167353 Run:The time cost of AIcoreEngine::CheckSupported is [299] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.769.761 [engine_place.cc:144][EVENT]167353 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.769.771 [engine_place.cc:144][EVENT]167353 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.769.856 [graph_manager.cc:3351][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [510] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.769.874 [graph_manager.cc:3364][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.769.943 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.769.961 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.770.122 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [140] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.770.165 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [30] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.770.213 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [34] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.770.247 [graph_manager.cc:3405][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [361] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.770.267 [graph_manager.cc:3412][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.866.744 [graph_manager.cc:3422][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [96460] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.866.804 [graph_manager.cc:3428][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.866.982 [graph_manager.cc:3467][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [155] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.001 [graph_manager.cc:3377][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [97117] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.018 [graph_manager.cc:1106][EVENT]167353 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [97693] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.031 [graph_manager.cc:1115][EVENT]167353 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:34.867.054 [graph_manager.cc:1130][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.085 [graph_manager.cc:1131][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.111 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.129 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.139 [graph_manager.cc:2837][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.233 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.867.245 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.867.254 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.867.263 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of BitcastPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.867.289 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.867.299 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.867.309 [graph_manager.cc:2864][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [154] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.321 [graph_manager.cc:2872][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.340 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.354 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.371 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.384 [compile_nodes_pass.cc:88][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.394 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.404 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.483 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [69] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.523 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.537 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.551 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.565 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.575 [graph_manager.cc:2927][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [238] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.587 [graph_manager.cc:2937][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.601 [graph_manager.cc:2943][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.612 [graph_manager.cc:2950][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.822 [graph_manager.cc:2958][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.855 [graph_manager.cc:1132][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [757] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.944 [graph_manager.cc:1135][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [66] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.867.975 [graph_manager.cc:2975][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.868.006 [graph_manager.cc:2981][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.868.021 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.868.030 [graph_manager.cc:2986][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.868.040 [graph_manager.cc:1136][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [81] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.868.179 [graph_manager.cc:3555][EVENT]167353 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [107] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.868.275 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.868.293 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.868.424 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [121] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.868.460 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.868.500 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [29] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.868.523 [graph_builder.cc:865][EVENT]167353 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [287] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:34.868.864 [logger.cc:1071] 167353 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:37:34.868.896 [task_generator.cc:804][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [91] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.868.955 [task_generator.cc:805][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [47] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.869.890 [task_generator.cc:814][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [919] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.869.907 [task_generator.cc:954][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1101] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.869.970 [task_generator.cc:967][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [36] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:34.869.989 [logger.cc:1084] 167353 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:37:34.871.777 [graph_manager.cc:1152][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3710] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.871.821 [graph_manager.cc:1164][EVENT]167353 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:34.871.873 [graph_manager.cc:1271][EVENT]167353 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [125905] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.871.886 [graph_manager.cc:1272][EVENT]167353 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:37:34.872.223 [atrace_api.c:93](tid:167353) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:37:34.872.242 [atrace_api.c:95](tid:167353) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:37:34.902.865 [graph_converter.cc:838][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [10520] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.903.058 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [133] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.903.796 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [711] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.904.044 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [219] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.904.067 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [245] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.904.265 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [185] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.904.316 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [32] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.904.355 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [25] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.904.615 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [247] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.904.728 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [90] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.904.745 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [108] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.904.782 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.904.814 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.904.847 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.904.957 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [99] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.905.050 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [82] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.905.062 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [94] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.905.096 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [24] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.905.127 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.905.143 [graph_converter.cc:849][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [2225] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.905.484 [graph_converter.cc:853][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [326] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.906.517 [graph_converter.cc:857][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [998] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.906.718 [graph_converter.cc:862][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [170] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.978.230 [graph_var_manager.cc:1424][EVENT]167354 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:34.978.328 [graph_manager.cc:1248][EVENT]167354 PreRun:PreRun start: graph node size 4, session id 14, graph id 13, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:34.978.567 [atrace_api.c:28](tid:167354) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:34.978.598 [trace_rb_log.c:84](tid:167354) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:34.978.611 [atrace_api.c:32](tid:167354) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:34.978.628 [client_manager.cpp:157][SetProfilingCallback][tid:167354] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:34.978.996 [parallel_partitioner.cc:165][EVENT]167354 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.979.032 [parallel_partitioner.cc:178][EVENT]167354 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.979.076 [graph_prepare.cc:1378][EVENT]167354 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.979.251 [graph_manager.cc:1050][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [191] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.979.274 [graph_manager.cc:1052][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.979.407 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.979.436 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.979.483 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.979.496 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.979.543 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.979.557 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.979.575 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.979.674 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.979.694 [graph_manager.cc:1054][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [406] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.979.950 [graph_manager.cc:1055][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [218] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.981.071 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:34.981.100 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.981.112 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.981.122 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferShapePass is [436] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.981.131 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.981.140 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:34.981.150 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.981.158 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.981.167 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.984.172 [graph_manager.cc:1056][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [4202] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.984.241 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.984.261 [graph_prepare.cc:1982][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [54] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.984.799 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:34.984.826 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.984.837 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.984.847 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferShapePass is [340] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.984.856 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.984.865 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:34.984.874 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.984.883 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.984.892 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.984.917 [graph_prepare.cc:1983][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [643] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.984.951 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.984.962 [graph_prepare.cc:1984][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.984.976 [graph_prepare.cc:1985][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.984.990 [graph_prepare.cc:1986][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.985.001 [graph_prepare.cc:1987][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.985.015 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.985.027 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.985.040 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.985.128 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.985.141 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.985.150 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.985.159 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.985.167 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.985.176 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.985.184 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.985.193 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.985.201 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.985.210 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.985.218 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.985.226 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.985.234 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.985.243 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.985.258 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.985.267 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:34.985.290 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.985.303 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.985.336 [graph_prepare.cc:1988][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [326] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.985.347 [graph_manager.cc:1065][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1141] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.998.465 [graph_manager.cc:1077][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13098] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.998.535 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:34.998.582 [graph_manager.cc:1080][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [81] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.009.467 [graph_manager.cc:1081][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [10868] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.009.511 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.009.527 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.009.539 [graph_manager.cc:1082][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.009.570 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.009.586 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.009.600 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.009.769 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [158] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.009.789 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.009.882 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [80] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.009.898 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.009.946 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.009.967 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.009.998 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.085 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [76] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.103 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.117 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.127 [graph_manager.cc:2700][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [562] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.353 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of EnterPass is [0] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.010.368 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.010.378 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.010.388 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.010.396 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.010.405 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CastRemovePass is [37] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.010.414 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.010.423 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.010.432 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [10] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.010.440 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.010.449 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.010.457 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.010.466 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.010.474 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [12] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.010.483 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.010.492 [graph_manager.cc:2741][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [347] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.502 [graph_manager.cc:2752][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.535 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.547 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.569 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.584 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.595 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.608 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.628 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.641 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.654 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.664 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.677 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.689 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.710 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.723 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.732 [graph_manager.cc:2810][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [203] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.775 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.010.787 [graph_manager.cc:2821][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [45] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.010.814 [graph_manager.cc:1087][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1255] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.011.373 [graph_manager.cc:1088][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [545] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.011.434 [graph_manager.cc:1089][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.011.457 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.011.476 [graph_manager.cc:1097][EVENT]167354 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:35.011.500 [graph_manager.cc:3325][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.021.958 [engine_place.cc:144][EVENT]167354 Run:The time cost of AIcoreEngine::CheckSupported is [10227] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.021.990 [engine_place.cc:144][EVENT]167354 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.022.001 [engine_place.cc:144][EVENT]167354 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.022.090 [graph_manager.cc:3351][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10565] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.022.108 [graph_manager.cc:3364][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.022.184 [engine_partitioner.cc:1139][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.022.215 [engine_partitioner.cc:1142][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.022.383 [engine_partitioner.cc:1148][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [158] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.022.422 [engine_partitioner.cc:1155][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [26] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.022.468 [engine_partitioner.cc:1164][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.022.502 [graph_manager.cc:3405][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [382] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.022.521 [graph_manager.cc:3412][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.136.171 [graph_manager.cc:3422][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [113636] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.136.217 [graph_manager.cc:3428][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.136.387 [graph_manager.cc:3467][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [150] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.136.406 [graph_manager.cc:3377][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [114286] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.136.423 [graph_manager.cc:1106][EVENT]167354 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [124931] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.136.435 [graph_manager.cc:1115][EVENT]167354 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:35.136.460 [graph_manager.cc:1130][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.136.492 [graph_manager.cc:1131][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.136.533 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.136.552 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.136.563 [graph_manager.cc:2837][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [41] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.136.696 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [26] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.136.709 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.136.719 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.136.729 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of BitcastPass is [3] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.136.738 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [7] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.136.747 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [15] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.136.756 [graph_manager.cc:2864][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [177] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.136.769 [graph_manager.cc:2872][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.136.788 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.136.802 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.136.819 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.136.833 [compile_nodes_pass.cc:88][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.136.842 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.136.853 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.136.959 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [97] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.137.008 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.137.022 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.137.037 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.137.060 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.137.071 [graph_manager.cc:2927][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [287] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.137.083 [graph_manager.cc:2937][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.137.099 [graph_manager.cc:2943][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.137.110 [graph_manager.cc:2950][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.137.327 [graph_manager.cc:2958][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [54] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.137.360 [graph_manager.cc:1132][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [842] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.137.442 [graph_manager.cc:1135][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [69] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.137.478 [graph_manager.cc:2975][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.137.511 [graph_manager.cc:2981][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.137.524 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.137.534 [graph_manager.cc:2986][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.137.543 [graph_manager.cc:1136][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [84] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.137.935 [graph_manager.cc:3555][EVENT]167354 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [352] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.138.069 [engine_partitioner.cc:1139][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.138.099 [engine_partitioner.cc:1142][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.138.241 [engine_partitioner.cc:1148][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [132] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.138.276 [engine_partitioner.cc:1155][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.138.317 [engine_partitioner.cc:1164][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [30] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.138.344 [graph_builder.cc:865][EVENT]167354 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [338] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:35.138.721 [logger.cc:1071] 167354 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:37:35.138.765 [task_generator.cc:804][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [90] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.138.845 [task_generator.cc:805][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [69] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.140.510 [task_generator.cc:814][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1650] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.140.526 [task_generator.cc:954][EVENT]167354 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1852] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.140.597 [task_generator.cc:967][EVENT]167354 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [38] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:35.140.624 [logger.cc:1084] 167354 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:37:35.142.132 [graph_manager.cc:1152][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4556] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.142.172 [graph_manager.cc:1164][EVENT]167354 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:35.142.208 [graph_manager.cc:1271][EVENT]167354 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [163298] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.142.220 [graph_manager.cc:1272][EVENT]167354 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:37:35.142.543 [atrace_api.c:93](tid:167354) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:37:35.142.561 [atrace_api.c:95](tid:167354) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:37:35.171.964 [graph_converter.cc:838][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [10915] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.172.182 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [164] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.173.664 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [1456] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.174.101 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [360] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.174.130 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [391] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.174.388 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [244] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.174.471 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [59] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.174.536 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [48] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.174.985 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [434] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.175.198 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [186] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.175.220 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [209] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.175.283 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [51] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.175.340 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [44] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.175.413 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [47] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.175.633 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [206] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.175.820 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [171] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.175.838 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [189] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.175.898 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [50] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.175.954 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [44] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.175.971 [graph_converter.cc:849][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [3959] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.176.664 [graph_converter.cc:853][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [681] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.178.588 [graph_converter.cc:857][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1899] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.178.964 [graph_converter.cc:862][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [345] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.251.271 [graph_var_manager.cc:1424][EVENT]167352 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:35.251.369 [graph_manager.cc:1248][EVENT]167352 PreRun:PreRun start: graph node size 4, session id 15, graph id 14, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:35.251.634 [atrace_api.c:28](tid:167352) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:35.251.662 [trace_rb_log.c:84](tid:167352) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:35.251.674 [atrace_api.c:32](tid:167352) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:35.251.691 [client_manager.cpp:157][SetProfilingCallback][tid:167352] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:35.252.091 [parallel_partitioner.cc:165][EVENT]167352 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.252.126 [parallel_partitioner.cc:178][EVENT]167352 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.252.172 [graph_prepare.cc:1378][EVENT]167352 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.252.343 [graph_manager.cc:1050][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [189] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.252.367 [graph_manager.cc:1052][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.252.502 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.252.558 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.252.605 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.252.619 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.252.664 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.252.677 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.252.695 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.252.795 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.252.814 [graph_manager.cc:1054][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [433] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.253.044 [graph_manager.cc:1055][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [217] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.254.205 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:35.254.235 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.254.246 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.254.256 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [473] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.254.265 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.254.274 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:35.254.283 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.254.291 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.254.300 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.257.157 [graph_manager.cc:1056][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [4093] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.257.225 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondRemovePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.257.243 [graph_prepare.cc:1982][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [52] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.257.817 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:35.257.846 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.257.868 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.257.878 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [371] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.257.888 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.257.897 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:35.257.906 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.257.914 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.257.923 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.257.950 [graph_prepare.cc:1983][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [694] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.257.975 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.257.986 [graph_prepare.cc:1984][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.258.000 [graph_prepare.cc:1985][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.258.015 [graph_prepare.cc:1986][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.258.026 [graph_prepare.cc:1987][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.258.041 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.258.052 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.258.067 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.258.157 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.258.169 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.258.178 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.258.186 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.258.195 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.258.203 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.258.212 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.258.231 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.258.240 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.258.249 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.258.258 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.258.266 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.258.274 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.258.283 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.258.291 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.258.299 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.258.323 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.258.336 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.258.367 [graph_prepare.cc:1988][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [332] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.258.380 [graph_manager.cc:1065][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1191] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.271.527 [graph_manager.cc:1077][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13127] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.271.596 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.271.644 [graph_manager.cc:1080][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [81] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.282.552 [graph_manager.cc:1081][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [10891] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.282.596 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.282.613 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.282.625 [graph_manager.cc:1082][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.282.656 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.282.670 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.282.697 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.282.844 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [137] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.282.861 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.282.950 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [79] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.282.965 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.014 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.036 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.055 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.141 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [75] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.158 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.171 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.180 [graph_manager.cc:2700][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [529] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.407 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of EnterPass is [4] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.283.423 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.283.433 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.283.442 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.283.451 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.283.459 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CastRemovePass is [38] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.283.468 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.283.476 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.283.485 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [9] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.283.493 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.283.512 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.283.521 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.283.529 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.283.538 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [9] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.283.546 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [3] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.283.556 [graph_manager.cc:2741][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [357] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.565 [graph_manager.cc:2752][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.590 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.602 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.623 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.638 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.651 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.664 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.683 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.697 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.710 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.721 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.733 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.745 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.766 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.779 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.788 [graph_manager.cc:2810][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [204] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.831 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.283.850 [graph_manager.cc:2821][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [53] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.283.876 [graph_manager.cc:1087][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1232] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.284.437 [graph_manager.cc:1088][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [546] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.284.498 [graph_manager.cc:1089][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.284.520 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.284.537 [graph_manager.cc:1097][EVENT]167352 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:35.284.559 [graph_manager.cc:3325][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.294.766 [engine_place.cc:144][EVENT]167352 Run:The time cost of AIcoreEngine::CheckSupported is [9931] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.294.798 [engine_place.cc:144][EVENT]167352 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.294.808 [engine_place.cc:144][EVENT]167352 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.294.897 [graph_manager.cc:3351][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10324] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.294.915 [graph_manager.cc:3364][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.294.991 [engine_partitioner.cc:1139][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [27] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.295.021 [engine_partitioner.cc:1142][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.295.186 [engine_partitioner.cc:1148][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [154] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.295.226 [engine_partitioner.cc:1155][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [26] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.295.274 [engine_partitioner.cc:1164][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.295.308 [graph_manager.cc:3405][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [380] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.295.326 [graph_manager.cc:3412][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.409.386 [graph_manager.cc:3422][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [114047] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.409.447 [graph_manager.cc:3428][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.409.617 [graph_manager.cc:3467][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [150] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.409.636 [graph_manager.cc:3377][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [114709] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.409.653 [graph_manager.cc:1106][EVENT]167352 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [125101] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.409.666 [graph_manager.cc:1115][EVENT]167352 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:35.409.740 [graph_manager.cc:1130][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [55] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.409.774 [graph_manager.cc:1131][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.409.803 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.409.822 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.409.832 [graph_manager.cc:2837][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [42] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.409.969 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [26] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.409.982 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.409.992 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.410.001 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.410.011 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [7] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.410.019 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [18] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:35.410.030 [graph_manager.cc:2864][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [180] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.042 [graph_manager.cc:2872][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.061 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.075 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.091 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.105 [compile_nodes_pass.cc:88][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.124 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.134 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.242 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [98] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.294 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [39] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.308 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.322 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.336 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.346 [graph_manager.cc:2927][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [288] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.359 [graph_manager.cc:2937][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.375 [graph_manager.cc:2943][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.386 [graph_manager.cc:2950][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.604 [graph_manager.cc:2958][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [54] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.636 [graph_manager.cc:1132][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [850] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.716 [graph_manager.cc:1135][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [67] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.753 [graph_manager.cc:2975][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.783 [graph_manager.cc:2981][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.797 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.806 [graph_manager.cc:2986][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.410.816 [graph_manager.cc:1136][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [84] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.411.139 [graph_manager.cc:3555][EVENT]167352 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [285] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.411.283 [engine_partitioner.cc:1139][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [32] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.411.313 [engine_partitioner.cc:1142][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.411.455 [engine_partitioner.cc:1148][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [131] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.411.491 [engine_partitioner.cc:1155][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.411.533 [engine_partitioner.cc:1164][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.411.559 [graph_builder.cc:865][EVENT]167352 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [339] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:35.411.935 [logger.cc:1071] 167352 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:37:35.411.966 [task_generator.cc:804][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [77] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.412.049 [task_generator.cc:805][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [70] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.413.664 [task_generator.cc:814][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1599] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.413.679 [task_generator.cc:954][EVENT]167352 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1790] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.413.801 [task_generator.cc:967][EVENT]167352 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [37] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:35.413.826 [logger.cc:1084] 167352 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:37:35.415.270 [graph_manager.cc:1152][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4423] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.415.310 [graph_manager.cc:1164][EVENT]167352 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:35.415.346 [graph_manager.cc:1271][EVENT]167352 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [163342] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.415.359 [graph_manager.cc:1272][EVENT]167352 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:37:35.415.678 [atrace_api.c:93](tid:167352) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:37:35.415.697 [atrace_api.c:95](tid:167352) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:37:35.443.694 [graph_converter.cc:838][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [10578] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.443.914 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [163] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.445.409 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [1468] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.445.836 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [397] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.445.865 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [428] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.446.143 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [252] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.446.228 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [61] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.446.295 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [49] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.446.749 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [439] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.446.962 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [186] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.446.985 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [210] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.447.049 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [54] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.447.106 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [45] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.447.164 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [45] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.447.384 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [207] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.447.575 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [174] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.447.592 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [191] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.447.653 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [51] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.447.709 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [44] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.447.727 [graph_converter.cc:849][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [3982] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.448.420 [graph_converter.cc:853][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [683] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.450.349 [graph_converter.cc:857][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1904] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.450.728 [graph_converter.cc:862][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [348] micro second. [INFO] HCCP(164046,python):2024-01-10-11:37:35.467.782 [ra_host.c:1761]tid:168560,ra_socket_white_list_add(1761) : Input parameters: phy_id[3], local_ip[3.0.0.0], num[1] [INFO] HCCP(164046,python):2024-01-10-11:37:35.468.032 [ra_host.c:825]tid:168560,ra_socket_batch_connect(825) : Input parameters: [0]th, phy_id[3], local_ip[3.0.0.0], remote_ip[0.0.0.0], tag:[8.92.9.85%enp189s0f0_60000_0_1704857845691656] [INFO] HCCP(164046,python):2024-01-10-11:37:35.468.050 [ra_host.c:825]tid:168560,ra_socket_batch_connect(825) : Input parameters: [1]th, phy_id[3], local_ip[3.0.0.0], remote_ip[2.0.0.0], tag:[8.92.9.85%enp189s0f0_60000_0_1704857845691656] [INFO] HCCP(164046,python):2024-01-10-11:37:35.509.327 [ra_host.c:825]tid:167354,ra_socket_batch_connect(825) : Input parameters: [0]th, phy_id[3], local_ip[3.0.0.0], remote_ip[1.0.0.0], tag:[HeartBeat_8.92.9.85/3_to_8.92.9.85/1] [INFO] HCCP(164046,python):2024-01-10-11:37:35.509.473 [ra_host.c:1761]tid:167354,ra_socket_white_list_add(1761) : Input parameters: phy_id[3], local_ip[3.0.0.0], num[1] [INFO] HCCL(164046,python):2024-01-10-11:37:35.515.420 [hccl_impl.cc:3258][167354]resource creation success, take time [48979]us, tag[AllReduce_8.92.9.85%enp189s0f0_60000_0_1704857845691656] [INFO] GE(164046,python):2024-01-10-11:37:35.588.864 [graph_var_manager.cc:1424][EVENT]167354 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:35.588.963 [graph_manager.cc:1248][EVENT]167354 PreRun:PreRun start: graph node size 3, session id 16, graph id 15, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:35.589.240 [atrace_api.c:28](tid:167354) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:35.589.267 [trace_rb_log.c:84](tid:167354) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:35.589.279 [atrace_api.c:32](tid:167354) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:35.589.297 [client_manager.cpp:157][SetProfilingCallback][tid:167354] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:35.589.674 [parallel_partitioner.cc:165][EVENT]167354 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.589.786 [parallel_partitioner.cc:178][EVENT]167354 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.589.831 [graph_prepare.cc:1378][EVENT]167354 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.589.961 [graph_manager.cc:1050][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [145] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.589.985 [graph_manager.cc:1052][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.590.103 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.590.131 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.590.176 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [34] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.590.190 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.590.234 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.590.248 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.590.264 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.590.357 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [3] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.590.377 [graph_manager.cc:1054][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [380] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.590.614 [graph_manager.cc:1055][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [223] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.591.485 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [5] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:35.591.513 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.591.524 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.591.534 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferShapePass is [266] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.591.543 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.591.552 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [5] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:35.591.561 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.591.570 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.591.579 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.593.471 [graph_manager.cc:1056][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2810] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.593.535 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.593.554 [graph_prepare.cc:1982][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [49] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.593.959 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:35.593.984 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [0] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.593.995 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.004 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferShapePass is [170] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.014 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.023 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [6] [INFO] GE(164046,python):2024-01-10-11:37:35.594.032 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.041 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.050 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.100 [graph_prepare.cc:1983][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [532] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.594.136 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.594.148 [graph_prepare.cc:1984][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.594.163 [graph_prepare.cc:1985][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.594.176 [graph_prepare.cc:1986][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.594.187 [graph_prepare.cc:1987][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.594.202 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.594.213 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.594.227 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.594.307 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.318 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.327 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.337 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.345 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.354 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.362 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.371 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.379 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.387 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.396 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.404 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.412 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.420 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.429 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.444 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.594.467 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.594.480 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.594.510 [graph_prepare.cc:1988][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [314] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.594.523 [graph_manager.cc:1065][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1019] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.606.437 [graph_manager.cc:1077][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [11894] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.606.536 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.606.587 [graph_manager.cc:1080][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [111] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.609.908 [graph_manager.cc:1081][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3304] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.609.950 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.609.964 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.609.975 [graph_manager.cc:1082][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [34] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.005 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.019 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.033 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.061 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.074 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.088 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.100 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.135 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [26] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.153 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.170 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.209 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.223 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.235 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.244 [graph_manager.cc:2700][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [244] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.344 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.610.358 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.610.367 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.610.376 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.610.385 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.610.393 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CastRemovePass is [8] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.610.402 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.610.411 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.610.419 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.610.428 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.610.436 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [6] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.610.445 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.610.453 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [8] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.610.462 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.610.470 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [4] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.610.479 [graph_manager.cc:2741][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [218] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.488 [graph_manager.cc:2752][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.510 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.528 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.546 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.560 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.570 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.583 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.601 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.614 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.626 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.637 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.650 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.662 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.679 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.691 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.700 [graph_manager.cc:2810][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [193] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.726 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.610.737 [graph_manager.cc:2821][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [29] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.765 [graph_manager.cc:1087][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [771] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.898 [graph_manager.cc:1088][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [120] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.934 [graph_manager.cc:1089][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.951 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.610.965 [graph_manager.cc:1097][EVENT]167354 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:35.610.985 [graph_manager.cc:3325][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.611.338 [engine_place.cc:144][EVENT]167354 Run:The time cost of AIcoreEngine::CheckSupported is [260] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.611.375 [engine_place.cc:144][EVENT]167354 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.611.385 [engine_place.cc:144][EVENT]167354 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.611.456 [graph_manager.cc:3351][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [456] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.611.472 [graph_manager.cc:3364][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.611.532 [engine_partitioner.cc:1139][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.611.548 [engine_partitioner.cc:1142][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.611.671 [engine_partitioner.cc:1148][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [113] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.611.710 [engine_partitioner.cc:1155][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [26] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.611.752 [engine_partitioner.cc:1164][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.611.783 [graph_manager.cc:3405][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [298] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.611.800 [graph_manager.cc:3412][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.209 [graph_manager.cc:3422][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [90396] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.260 [graph_manager.cc:3428][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.400 [graph_manager.cc:3467][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [118] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.417 [graph_manager.cc:3377][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [90934] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.433 [graph_manager.cc:1106][EVENT]167354 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [91454] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.445 [graph_manager.cc:1115][EVENT]167354 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:35.702.469 [graph_manager.cc:1130][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.500 [graph_manager.cc:1131][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.523 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.553 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.563 [graph_manager.cc:2837][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [48] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.641 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.702.654 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.702.663 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.702.672 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.702.680 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [4] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.702.689 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [3] [INFO] GE(164046,python):2024-01-10-11:37:35.702.699 [graph_manager.cc:2864][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [119] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.711 [graph_manager.cc:2872][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.730 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.743 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.758 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.772 [compile_nodes_pass.cc:88][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.782 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.792 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.863 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [61] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.902 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [27] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.915 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.928 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.942 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.959 [graph_manager.cc:2927][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [232] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.972 [graph_manager.cc:2937][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.985 [graph_manager.cc:2943][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.702.997 [graph_manager.cc:2950][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.703.194 [graph_manager.cc:2958][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [33] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.703.224 [graph_manager.cc:1132][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [712] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.703.301 [graph_manager.cc:1135][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [64] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.703.332 [graph_manager.cc:2975][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.703.364 [graph_manager.cc:2981][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.703.378 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.703.387 [graph_manager.cc:2986][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.703.396 [graph_manager.cc:1136][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [80] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.703.517 [graph_manager.cc:3555][EVENT]167354 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [91] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.703.602 [engine_partitioner.cc:1139][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.703.619 [engine_partitioner.cc:1142][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.703.715 [engine_partitioner.cc:1148][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [86] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.703.749 [engine_partitioner.cc:1155][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.703.791 [engine_partitioner.cc:1164][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.703.814 [graph_builder.cc:865][EVENT]167354 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [244] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:35.704.089 [logger.cc:1071] 167354 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:37:35.704.120 [task_generator.cc:804][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [76] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.704.185 [task_generator.cc:805][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [42] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.704.967 [task_generator.cc:814][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [767] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.704.982 [task_generator.cc:954][EVENT]167354 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [938] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.705.047 [task_generator.cc:967][EVENT]167354 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [37] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:35.705.066 [logger.cc:1084] 167354 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:37:35.706.424 [graph_manager.cc:1152][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3004] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.706.462 [graph_manager.cc:1164][EVENT]167354 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:35.706.500 [graph_manager.cc:1271][EVENT]167354 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [116908] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.706.512 [graph_manager.cc:1272][EVENT]167354 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:37:35.706.825 [atrace_api.c:93](tid:167354) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:37:35.706.842 [atrace_api.c:95](tid:167354) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:37:35.730.117 [graph_converter.cc:838][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [7855] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.730.303 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [129] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.730.926 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [598] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.731.139 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [186] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.731.162 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [211] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.731.347 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [172] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.731.392 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [26] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.731.426 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.731.649 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [211] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.731.748 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [78] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.731.764 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [94] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.731.797 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.731.825 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.731.855 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.731.960 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [81] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.732.041 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [69] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.732.053 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [82] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.732.082 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.732.109 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.732.122 [graph_converter.cc:849][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1956] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.732.400 [graph_converter.cc:853][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [268] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.733.225 [graph_converter.cc:857][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [808] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.733.390 [graph_converter.cc:862][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [137] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.824.313 [graph_var_manager.cc:1424][EVENT]167355 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:35.824.408 [graph_manager.cc:1248][EVENT]167355 PreRun:PreRun start: graph node size 4, session id 17, graph id 16, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:35.825.305 [atrace_api.c:28](tid:167355) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:35.825.379 [trace_rb_log.c:84](tid:167355) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:35.825.395 [atrace_api.c:32](tid:167355) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:35.825.415 [client_manager.cpp:157][SetProfilingCallback][tid:167355] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:35.826.371 [parallel_partitioner.cc:165][EVENT]167355 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.826.415 [parallel_partitioner.cc:178][EVENT]167355 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.826.463 [graph_prepare.cc:1378][EVENT]167355 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.827.124 [graph_manager.cc:1050][EVENT]167355 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [678] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.827.155 [graph_manager.cc:1052][EVENT]167355 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.827.293 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.827.322 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.827.389 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.827.403 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.827.451 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.827.464 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.827.483 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.827.582 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.827.603 [graph_manager.cc:1054][EVENT]167355 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [435] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.827.833 [graph_manager.cc:1055][EVENT]167355 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [215] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.828.847 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:35.828.877 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.828.889 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.828.899 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of InferShapePass is [320] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.828.908 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.828.917 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:35.828.926 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.828.934 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.828.943 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of InferValuePass is [10] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.830.970 [graph_manager.cc:1056][EVENT]167355 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3117] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.831.040 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.060 [graph_prepare.cc:1982][EVENT]167355 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [52] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.831.511 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:35.831.537 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.551 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.572 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of InferShapePass is [255] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.582 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.591 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:35.831.600 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.609 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.617 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.643 [graph_prepare.cc:1983][EVENT]167355 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [570] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.831.666 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.831.677 [graph_prepare.cc:1984][EVENT]167355 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.831.691 [graph_prepare.cc:1985][EVENT]167355 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.831.709 [graph_prepare.cc:1986][EVENT]167355 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.831.721 [graph_prepare.cc:1987][EVENT]167355 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.831.736 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.831.747 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.831.761 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.831.850 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.863 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.872 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.881 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.890 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.898 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.907 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.915 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.932 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.941 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.949 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.958 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.966 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.975 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.983 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.831.991 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.832.013 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.832.025 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.832.058 [graph_prepare.cc:1988][EVENT]167355 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [327] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.832.070 [graph_manager.cc:1065][EVENT]167355 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1064] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.844.449 [graph_manager.cc:1077][EVENT]167355 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12359] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.844.523 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.844.573 [graph_manager.cc:1080][EVENT]167355 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [87] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.164 [graph_manager.cc:1081][EVENT]167355 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [8574] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.208 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.224 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.236 [graph_manager.cc:1082][EVENT]167355 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.267 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.281 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.295 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.404 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [87] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.423 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.466 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [33] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.484 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.523 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.544 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.574 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.607 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.622 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.634 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.643 [graph_manager.cc:2700][EVENT]167355 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [381] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.780 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of EnterPass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.853.797 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.853.807 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.853.816 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.853.825 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.853.835 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of CastRemovePass is [22] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.853.843 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.853.852 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.853.860 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.853.871 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.853.883 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.853.904 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.853.914 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.853.922 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.853.931 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.853.940 [graph_manager.cc:2741][EVENT]167355 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [279] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.950 [graph_manager.cc:2752][EVENT]167355 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.972 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.853.983 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.000 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.015 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.027 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.039 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.057 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.073 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.086 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.100 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.113 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.124 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.143 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.158 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.171 [graph_manager.cc:2810][EVENT]167355 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [203] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.200 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.854.211 [graph_manager.cc:2821][EVENT]167355 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.246 [graph_manager.cc:1087][EVENT]167355 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [991] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.384 [graph_manager.cc:1088][EVENT]167355 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [124] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.424 [graph_manager.cc:1089][EVENT]167355 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.442 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.458 [graph_manager.cc:1097][EVENT]167355 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:35.854.478 [graph_manager.cc:3325][EVENT]167355 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.870 [engine_place.cc:144][EVENT]167355 Run:The time cost of AIcoreEngine::CheckSupported is [293] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.898 [engine_place.cc:144][EVENT]167355 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.908 [engine_place.cc:144][EVENT]167355 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.854.986 [graph_manager.cc:3351][EVENT]167355 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [495] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.855.004 [graph_manager.cc:3364][EVENT]167355 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.855.067 [engine_partitioner.cc:1139][EVENT]167355 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.855.084 [engine_partitioner.cc:1142][EVENT]167355 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.855.250 [engine_partitioner.cc:1148][EVENT]167355 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [156] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.855.293 [engine_partitioner.cc:1155][EVENT]167355 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [29] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.855.347 [engine_partitioner.cc:1164][EVENT]167355 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [43] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.855.384 [graph_manager.cc:3405][EVENT]167355 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [367] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.855.402 [graph_manager.cc:3412][EVENT]167355 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.870.916 [graph_manager.cc:3422][EVENT]167355 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [15499] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.870.954 [graph_manager.cc:3428][EVENT]167355 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.099 [graph_manager.cc:3467][EVENT]167355 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [115] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.120 [graph_manager.cc:3377][EVENT]167355 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [16104] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.135 [graph_manager.cc:1106][EVENT]167355 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [16663] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.148 [graph_manager.cc:1115][EVENT]167355 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:35.871.170 [graph_manager.cc:1130][EVENT]167355 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.202 [graph_manager.cc:1131][EVENT]167355 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.225 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.241 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.251 [graph_manager.cc:2837][EVENT]167355 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.334 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.871.346 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.871.355 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.871.364 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.871.373 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.871.381 [base_pass.cc:339][EVENT]167355 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [8] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.871.391 [graph_manager.cc:2864][EVENT]167355 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [123] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.403 [graph_manager.cc:2872][EVENT]167355 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.421 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.435 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.449 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.462 [compile_nodes_pass.cc:88][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.481 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.492 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.577 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [76] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.606 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.619 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.632 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.645 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.655 [graph_manager.cc:2927][EVENT]167355 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [236] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.667 [graph_manager.cc:2937][EVENT]167355 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.680 [graph_manager.cc:2943][EVENT]167355 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.691 [graph_manager.cc:2950][EVENT]167355 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.886 [graph_manager.cc:2958][EVENT]167355 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.920 [graph_manager.cc:1132][EVENT]167355 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [705] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.871.991 [graph_manager.cc:1135][EVENT]167355 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [56] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.872.028 [graph_manager.cc:2975][EVENT]167355 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.872.059 [graph_manager.cc:2981][EVENT]167355 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.872.073 [pass_manager.cc:82][EVENT]167355 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.872.083 [graph_manager.cc:2986][EVENT]167355 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.872.092 [graph_manager.cc:1136][EVENT]167355 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [86] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.872.227 [graph_manager.cc:3555][EVENT]167355 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [103] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.872.317 [engine_partitioner.cc:1139][EVENT]167355 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.872.345 [engine_partitioner.cc:1142][EVENT]167355 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.872.475 [engine_partitioner.cc:1148][EVENT]167355 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [120] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.872.507 [engine_partitioner.cc:1155][EVENT]167355 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.872.547 [engine_partitioner.cc:1164][EVENT]167355 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [30] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.872.569 [graph_builder.cc:865][EVENT]167355 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [286] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:35.873.002 [logger.cc:1071] 167355 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:37:35.873.038 [task_generator.cc:804][EVENT]167355 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [168] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.873.106 [task_generator.cc:805][EVENT]167355 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [56] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.873.851 [task_generator.cc:814][EVENT]167355 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [729] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.873.868 [task_generator.cc:954][EVENT]167355 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [997] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.873.927 [task_generator.cc:967][EVENT]167355 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [33] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:35.873.944 [logger.cc:1084] 167355 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:37:35.875.150 [graph_manager.cc:1152][EVENT]167355 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3033] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.875.184 [graph_manager.cc:1164][EVENT]167355 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:35.875.218 [graph_manager.cc:1271][EVENT]167355 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [48937] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.875.229 [graph_manager.cc:1272][EVENT]167355 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:37:35.875.537 [atrace_api.c:93](tid:167355) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:37:35.875.556 [atrace_api.c:95](tid:167355) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:37:35.886.991 [graph_converter.cc:838][EVENT]167355 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [3490] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.887.154 [base_optimizer.cc:70][EVENT]167355 Run:[GEPERFTRACE] The time cost of ZeroCopy is [116] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.887.641 [base_optimizer.cc:70][EVENT]167355 Run:[GEPERFTRACE] The time cost of CEM is [463] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.887.848 [copy_flow_launch_fuse.cc:395][EVENT]167355 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [183] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.887.869 [base_optimizer.cc:70][EVENT]167355 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [205] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.888.087 [base_optimizer.cc:70][EVENT]167355 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [206] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.888.140 [base_optimizer.cc:70][EVENT]167355 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.888.172 [base_optimizer.cc:70][EVENT]167355 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.888.370 [base_optimizer.cc:70][EVENT]167355 Run:[GEPERFTRACE] The time cost of CEM is [186] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.888.453 [copy_flow_launch_fuse.cc:395][EVENT]167355 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [65] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.888.466 [base_optimizer.cc:70][EVENT]167355 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [79] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.888.496 [base_optimizer.cc:70][EVENT]167355 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.888.521 [base_optimizer.cc:70][EVENT]167355 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.888.547 [base_optimizer.cc:70][EVENT]167355 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.888.623 [base_optimizer.cc:70][EVENT]167355 Run:[GEPERFTRACE] The time cost of CEM is [67] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.888.690 [copy_flow_launch_fuse.cc:395][EVENT]167355 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [56] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.888.701 [base_optimizer.cc:70][EVENT]167355 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [68] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.888.727 [base_optimizer.cc:70][EVENT]167355 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.888.751 [base_optimizer.cc:70][EVENT]167355 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.888.764 [graph_converter.cc:849][EVENT]167355 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1733] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.888.984 [graph_converter.cc:853][EVENT]167355 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [210] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.889.685 [graph_converter.cc:857][EVENT]167355 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [687] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.889.854 [graph_converter.cc:862][EVENT]167355 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [127] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.959.952 [graph_var_manager.cc:1424][EVENT]167352 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:35.960.037 [graph_manager.cc:1248][EVENT]167352 PreRun:PreRun start: graph node size 4, session id 18, graph id 17, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:35.963.239 [atrace_api.c:28](tid:167352) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:35.963.489 [trace_rb_log.c:84](tid:167352) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:35.963.510 [atrace_api.c:32](tid:167352) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:35.963.542 [client_manager.cpp:157][SetProfilingCallback][tid:167352] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:35.964.901 [parallel_partitioner.cc:165][EVENT]167352 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.964.956 [parallel_partitioner.cc:178][EVENT]167352 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.965.001 [graph_prepare.cc:1378][EVENT]167352 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.965.406 [graph_manager.cc:1050][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [420] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.965.429 [graph_manager.cc:1052][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.965.556 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.965.580 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.965.650 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [58] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.965.663 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.965.948 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [42] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.966.024 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.966.054 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.966.248 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.966.278 [graph_manager.cc:1054][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [837] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.966.566 [graph_manager.cc:1055][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [272] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.967.942 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:35.967.972 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.967.985 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.967.997 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [407] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.968.010 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [17] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.968.019 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:35.968.031 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.968.040 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.968.065 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.970.366 [graph_manager.cc:1056][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3778] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.970.440 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.970.459 [graph_prepare.cc:1982][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [54] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.970.941 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:35.970.969 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.970.980 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.970.990 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [264] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.970.999 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.971.008 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:35.971.017 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.971.026 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.971.034 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.971.061 [graph_prepare.cc:1983][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [588] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.971.085 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.971.095 [graph_prepare.cc:1984][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.971.110 [graph_prepare.cc:1985][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.971.126 [graph_prepare.cc:1986][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.971.137 [graph_prepare.cc:1987][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.971.152 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.971.164 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.971.177 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.971.280 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.971.293 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.971.302 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.971.311 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.971.320 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.971.328 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.971.337 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.971.345 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.971.354 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of StopGradientPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.971.362 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.971.370 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.971.379 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.971.387 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.971.395 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [7] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.971.404 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.971.412 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.971.435 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.971.450 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.971.486 [graph_prepare.cc:1988][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [340] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.971.502 [graph_manager.cc:1065][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1098] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.985.490 [graph_manager.cc:1077][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13964] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.985.570 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.985.619 [graph_manager.cc:1080][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [86] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.991.720 [graph_manager.cc:1081][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [6061] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.991.770 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.991.785 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.991.797 [graph_manager.cc:1082][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.991.828 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.991.843 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.991.857 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.991.886 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.991.900 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.991.915 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.991.928 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.991.969 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [32] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.991.990 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.010 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.037 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.052 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.064 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.074 [graph_manager.cc:2700][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [252] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.192 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of EnterPass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.992.206 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.992.216 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.992.235 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.992.245 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.992.254 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CastRemovePass is [9] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.992.263 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.992.272 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.992.280 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.992.288 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.992.297 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.992.305 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.992.313 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.992.322 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.992.330 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.992.340 [graph_manager.cc:2741][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [247] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.349 [graph_manager.cc:2752][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.373 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.385 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.401 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.416 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.427 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.440 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.459 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.474 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.487 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.503 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.517 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.528 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.548 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.561 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.570 [graph_manager.cc:2810][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [202] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.600 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:35.992.611 [graph_manager.cc:2821][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.639 [graph_manager.cc:1087][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [825] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.779 [graph_manager.cc:1088][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [126] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.820 [graph_manager.cc:1089][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.837 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.992.854 [graph_manager.cc:1097][EVENT]167352 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:35.992.877 [graph_manager.cc:3325][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.993.405 [engine_place.cc:144][EVENT]167352 Run:The time cost of AIcoreEngine::CheckSupported is [422] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.993.435 [engine_place.cc:144][EVENT]167352 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.993.444 [engine_place.cc:144][EVENT]167352 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.993.524 [graph_manager.cc:3351][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [634] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.993.542 [graph_manager.cc:3364][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.993.617 [engine_partitioner.cc:1139][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.993.635 [engine_partitioner.cc:1142][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.994.072 [engine_partitioner.cc:1148][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [426] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.994.164 [engine_partitioner.cc:1155][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [47] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.994.222 [engine_partitioner.cc:1164][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [44] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.994.254 [graph_manager.cc:3405][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [701] micro second. [INFO] GE(164046,python):2024-01-10-11:37:35.994.282 [graph_manager.cc:3412][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.014.388 [graph_manager.cc:3422][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [20090] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.014.441 [graph_manager.cc:3428][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.014.592 [graph_manager.cc:3467][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [130] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.014.610 [graph_manager.cc:3377][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [21058] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.014.627 [graph_manager.cc:1106][EVENT]167352 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [21758] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.014.640 [graph_manager.cc:1115][EVENT]167352 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:36.014.663 [graph_manager.cc:1130][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.014.695 [graph_manager.cc:1131][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.014.718 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.014.734 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.014.744 [graph_manager.cc:2837][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [34] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.014.839 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.014.852 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.014.861 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.014.870 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of BitcastPass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.014.880 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.014.888 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [12] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.014.913 [graph_manager.cc:2864][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [153] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.014.924 [graph_manager.cc:2872][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.014.944 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.014.958 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.014.972 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.014.984 [compile_nodes_pass.cc:88][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.014.994 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.015.004 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.015.076 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [64] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.015.112 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.015.125 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.015.138 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.015.151 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.015.161 [graph_manager.cc:2927][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [221] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.015.173 [graph_manager.cc:2937][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.015.187 [graph_manager.cc:2943][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.015.198 [graph_manager.cc:2950][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.015.434 [graph_manager.cc:2958][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.015.462 [graph_manager.cc:1132][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [754] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.015.543 [graph_manager.cc:1135][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [67] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.015.582 [graph_manager.cc:2975][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.015.788 [graph_manager.cc:2981][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [190] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.015.824 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.015.836 [graph_manager.cc:2986][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.015.847 [graph_manager.cc:1136][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [279] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.016.035 [graph_manager.cc:3555][EVENT]167352 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [133] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.016.140 [engine_partitioner.cc:1139][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.016.156 [engine_partitioner.cc:1142][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.016.312 [engine_partitioner.cc:1148][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [145] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.016.356 [engine_partitioner.cc:1155][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [29] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.016.408 [engine_partitioner.cc:1164][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [40] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.016.441 [graph_builder.cc:865][EVENT]167352 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [342] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:36.016.858 [logger.cc:1071] 167352 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:37:36.016.893 [task_generator.cc:804][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [168] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.016.959 [task_generator.cc:805][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [54] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.018.012 [task_generator.cc:814][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1038] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.018.029 [task_generator.cc:954][EVENT]167352 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1304] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.018.092 [task_generator.cc:967][EVENT]167352 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [37] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:36.018.110 [logger.cc:1084] 167352 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:37:36.018.967 [graph_manager.cc:1152][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3069] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.019.002 [graph_manager.cc:1164][EVENT]167352 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:36.019.047 [graph_manager.cc:1271][EVENT]167352 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [54238] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.019.075 [graph_manager.cc:1272][EVENT]167352 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:37:36.019.437 [atrace_api.c:93](tid:167352) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:37:36.019.457 [atrace_api.c:95](tid:167352) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:37:36.041.122 [graph_converter.cc:838][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [6072] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.041.312 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [139] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.042.001 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [664] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.042.259 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [230] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.042.281 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [253] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.042.514 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [220] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.042.816 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [281] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.042.904 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [48] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.043.293 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [374] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.043.423 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [111] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.043.438 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [127] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.043.471 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.043.497 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.043.524 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.043.601 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [67] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.043.669 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [56] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.043.680 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [68] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.043.706 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.043.730 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.043.745 [graph_converter.cc:849][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [2578] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.044.151 [graph_converter.cc:853][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [396] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.044.978 [graph_converter.cc:857][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [811] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.045.144 [graph_converter.cc:862][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [126] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.126.592 [graph_var_manager.cc:1424][EVENT]167353 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:36.126.690 [graph_manager.cc:1248][EVENT]167353 PreRun:PreRun start: graph node size 4, session id 19, graph id 18, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:36.127.282 [atrace_api.c:28](tid:167353) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:36.127.337 [trace_rb_log.c:84](tid:167353) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:36.127.350 [atrace_api.c:32](tid:167353) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:36.127.369 [client_manager.cpp:157][SetProfilingCallback][tid:167353] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:36.128.082 [parallel_partitioner.cc:165][EVENT]167353 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.128.122 [parallel_partitioner.cc:178][EVENT]167353 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.128.168 [graph_prepare.cc:1378][EVENT]167353 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.128.591 [graph_manager.cc:1050][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [441] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.128.618 [graph_manager.cc:1052][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.128.756 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.128.785 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.128.834 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.128.847 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.128.899 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.128.913 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.128.929 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.129.027 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.129.049 [graph_manager.cc:1054][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [418] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.129.291 [graph_manager.cc:1055][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [229] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.130.392 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:36.130.422 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.130.433 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.130.443 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [320] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.130.452 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.130.461 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [4] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:36.130.471 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.130.480 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.130.488 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [10] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.132.512 [graph_manager.cc:1056][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3175] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.132.581 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.132.599 [graph_prepare.cc:1982][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.133.062 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:36.133.087 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.099 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.109 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [260] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.118 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.127 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:36.133.136 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.144 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.153 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.178 [graph_prepare.cc:1983][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [567] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.133.201 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.133.223 [graph_prepare.cc:1984][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.133.238 [graph_prepare.cc:1985][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.133.252 [graph_prepare.cc:1986][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.133.262 [graph_prepare.cc:1987][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.133.277 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.133.289 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.133.302 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.133.392 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.404 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.413 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.422 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.431 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DropOutPass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.439 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.448 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.457 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.465 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.473 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.482 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.490 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SnapshotPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.498 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.506 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [7] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.515 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.523 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.133.552 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.133.565 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.133.597 [graph_prepare.cc:1988][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [325] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.133.609 [graph_manager.cc:1065][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1064] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.146.706 [graph_manager.cc:1077][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13078] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.146.776 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.146.826 [graph_manager.cc:1080][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [83] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.060 [graph_manager.cc:1081][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [8218] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.106 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.120 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.132 [graph_manager.cc:1082][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.163 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.178 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.192 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.287 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [82] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.306 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.351 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [34] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.367 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.405 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [29] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.425 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.451 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.494 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.512 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.527 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.536 [graph_manager.cc:2700][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [379] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.657 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.155.671 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AddNPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.155.681 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.155.690 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.155.699 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.155.708 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CastRemovePass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.155.717 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.155.728 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.155.737 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.155.745 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.155.754 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.155.762 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.155.770 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.155.778 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.155.787 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.155.796 [graph_manager.cc:2741][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [242] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.805 [graph_manager.cc:2752][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.829 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.843 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.871 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.887 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.901 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.914 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.935 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.949 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.962 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.972 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.984 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.155.995 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.156.016 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.156.031 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.156.040 [graph_manager.cc:2810][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [214] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.156.069 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.156.081 [graph_manager.cc:2821][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [32] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.156.111 [graph_manager.cc:1087][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [960] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.156.249 [graph_manager.cc:1088][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [122] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.156.292 [graph_manager.cc:1089][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.156.311 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.156.326 [graph_manager.cc:1097][EVENT]167353 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:36.156.347 [graph_manager.cc:3325][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.156.729 [engine_place.cc:144][EVENT]167353 Run:The time cost of AIcoreEngine::CheckSupported is [281] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.156.767 [engine_place.cc:144][EVENT]167353 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.156.778 [engine_place.cc:144][EVENT]167353 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.156.858 [graph_manager.cc:3351][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [497] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.156.875 [graph_manager.cc:3364][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.156.942 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.156.960 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.157.119 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [150] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.157.163 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [30] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.157.215 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.157.248 [graph_manager.cc:3405][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [360] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.157.266 [graph_manager.cc:3412][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.141 [graph_manager.cc:3422][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [16861] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.182 [graph_manager.cc:3428][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.316 [graph_manager.cc:3467][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [114] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.335 [graph_manager.cc:3377][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [17448] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.350 [graph_manager.cc:1106][EVENT]167353 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [18010] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.363 [graph_manager.cc:1115][EVENT]167353 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:36.174.386 [graph_manager.cc:1130][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.417 [graph_manager.cc:1131][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.440 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.476 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.486 [graph_manager.cc:2837][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [55] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.569 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.174.582 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.174.592 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.174.600 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.174.609 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.174.618 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [7] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.174.628 [graph_manager.cc:2864][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [126] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.640 [graph_manager.cc:2872][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.660 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.673 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.688 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.701 [compile_nodes_pass.cc:88][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.711 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.721 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.800 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [70] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.826 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.838 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.850 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.863 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.871 [graph_manager.cc:2927][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [215] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.891 [graph_manager.cc:2937][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.906 [graph_manager.cc:2943][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.174.917 [graph_manager.cc:2950][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.175.143 [graph_manager.cc:2958][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.175.174 [graph_manager.cc:1132][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [744] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.175.247 [graph_manager.cc:1135][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [59] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.175.278 [graph_manager.cc:2975][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.175.315 [graph_manager.cc:2981][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.175.329 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.175.339 [graph_manager.cc:2986][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.175.349 [graph_manager.cc:1136][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [86] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.175.484 [graph_manager.cc:3555][EVENT]167353 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [103] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.175.574 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.175.591 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.175.711 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [109] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.175.743 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.175.780 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [27] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.175.802 [graph_builder.cc:865][EVENT]167353 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [262] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:36.176.100 [logger.cc:1071] 167353 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:37:36.176.130 [task_generator.cc:804][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [72] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.176.202 [task_generator.cc:805][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [49] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.176.889 [task_generator.cc:814][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [672] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.176.903 [task_generator.cc:954][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [845] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.176.960 [task_generator.cc:967][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [32] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:36.176.978 [logger.cc:1084] 167353 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:37:36.177.576 [graph_manager.cc:1152][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2201] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.177.609 [graph_manager.cc:1164][EVENT]167353 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:36.177.645 [graph_manager.cc:1271][EVENT]167353 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [49654] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.177.657 [graph_manager.cc:1272][EVENT]167353 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:37:36.177.983 [atrace_api.c:93](tid:167353) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:37:36.178.002 [atrace_api.c:95](tid:167353) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:37:36.189.572 [graph_converter.cc:838][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [3728] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.189.758 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [140] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.190.254 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [470] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.190.464 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [187] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.190.486 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [210] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.190.701 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [203] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.190.743 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.190.775 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.190.974 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [185] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.191.061 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [66] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.191.075 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [81] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.191.105 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.191.131 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.191.157 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.191.234 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [68] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.191.314 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [58] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.191.327 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [70] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.191.353 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.191.377 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.191.390 [graph_converter.cc:849][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1776] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.191.611 [graph_converter.cc:853][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [212] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.192.332 [graph_converter.cc:857][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [702] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.192.483 [graph_converter.cc:862][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [124] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.277.481 [graph_var_manager.cc:1424][EVENT]167356 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:36.277.571 [graph_manager.cc:1248][EVENT]167356 PreRun:PreRun start: graph node size 4, session id 20, graph id 19, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:36.278.489 [atrace_api.c:28](tid:167356) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:36.278.562 [trace_rb_log.c:84](tid:167356) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:36.278.579 [atrace_api.c:32](tid:167356) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:36.278.598 [client_manager.cpp:157][SetProfilingCallback][tid:167356] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:36.279.485 [parallel_partitioner.cc:165][EVENT]167356 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.279.526 [parallel_partitioner.cc:178][EVENT]167356 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.279.573 [graph_prepare.cc:1378][EVENT]167356 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.280.257 [graph_manager.cc:1050][EVENT]167356 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [700] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.280.289 [graph_manager.cc:1052][EVENT]167356 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.280.424 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.280.454 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.280.501 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.280.534 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.280.582 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.280.595 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.280.613 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.280.717 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.280.738 [graph_manager.cc:1054][EVENT]167356 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [437] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.280.974 [graph_manager.cc:1055][EVENT]167356 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [223] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.282.165 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:36.282.195 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.282.206 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.282.216 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of InferShapePass is [485] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.282.226 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.282.235 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:36.282.243 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.282.252 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.282.261 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.285.133 [graph_manager.cc:1056][EVENT]167356 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [4139] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.285.203 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.285.222 [graph_prepare.cc:1982][EVENT]167356 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [52] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.285.794 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:36.285.822 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.285.833 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.285.843 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of InferShapePass is [333] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.285.863 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.285.872 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:37:36.285.881 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.285.890 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [47] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.285.898 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.285.924 [graph_prepare.cc:1983][EVENT]167356 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [688] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.285.946 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.285.958 [graph_prepare.cc:1984][EVENT]167356 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.285.972 [graph_prepare.cc:1985][EVENT]167356 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.285.989 [graph_prepare.cc:1986][EVENT]167356 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.286.002 [graph_prepare.cc:1987][EVENT]167356 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.286.018 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.286.030 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.286.043 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.286.132 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of EnterPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.286.147 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.286.157 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.286.166 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.286.174 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.286.183 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.286.191 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.286.200 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.286.216 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of StopGradientPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.286.225 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.286.234 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.286.243 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.286.252 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.286.260 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.286.268 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.286.277 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:37:36.286.299 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.286.311 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.286.343 [graph_prepare.cc:1988][EVENT]167356 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [330] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.286.355 [graph_manager.cc:1065][EVENT]167356 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1187] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.299.351 [graph_manager.cc:1077][EVENT]167356 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12976] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.299.448 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.299.498 [graph_manager.cc:1080][EVENT]167356 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [109] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.310.226 [graph_manager.cc:1081][EVENT]167356 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [10710] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.310.270 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.310.286 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.310.298 [graph_manager.cc:1082][EVENT]167356 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.310.329 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.310.345 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.310.359 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.310.515 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [135] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.310.533 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.310.626 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [81] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.310.642 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.310.689 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.310.711 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.310.731 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.310.818 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [76] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.310.835 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.310.848 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.310.858 [graph_manager.cc:2700][EVENT]167356 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [534] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.311.089 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.311.105 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.311.116 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.311.125 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.311.135 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.311.144 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of CastRemovePass is [39] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.311.152 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.311.161 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.311.169 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [12] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.311.178 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.311.187 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.311.205 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.311.215 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.311.223 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [11] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.311.232 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.311.241 [graph_manager.cc:2741][EVENT]167356 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [366] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.311.251 [graph_manager.cc:2752][EVENT]167356 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.311.274 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.311.286 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.311.308 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.311.323 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.311.335 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.311.347 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.311.367 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.311.382 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.311.394 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.311.404 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.311.416 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.311.429 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.311.450 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.311.463 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.311.472 [graph_manager.cc:2810][EVENT]167356 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [202] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.311.515 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.311.525 [graph_manager.cc:2821][EVENT]167356 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [45] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.311.559 [graph_manager.cc:1087][EVENT]167356 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1243] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.312.128 [graph_manager.cc:1088][EVENT]167356 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [554] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.312.189 [graph_manager.cc:1089][EVENT]167356 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [30] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.312.212 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.312.231 [graph_manager.cc:1097][EVENT]167356 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:36.312.254 [graph_manager.cc:3325][EVENT]167356 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.321.573 [engine_place.cc:144][EVENT]167356 Run:The time cost of AIcoreEngine::CheckSupported is [9086] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.321.605 [engine_place.cc:144][EVENT]167356 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.321.616 [engine_place.cc:144][EVENT]167356 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.321.733 [graph_manager.cc:3351][EVENT]167356 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [9448] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.321.753 [graph_manager.cc:3364][EVENT]167356 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.321.829 [engine_partitioner.cc:1139][EVENT]167356 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.321.860 [engine_partitioner.cc:1142][EVENT]167356 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.322.037 [engine_partitioner.cc:1148][EVENT]167356 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [167] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.322.077 [engine_partitioner.cc:1155][EVENT]167356 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [26] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.322.127 [engine_partitioner.cc:1164][EVENT]167356 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.322.165 [graph_manager.cc:3405][EVENT]167356 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [399] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.322.184 [graph_manager.cc:3412][EVENT]167356 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.435.991 [graph_manager.cc:3422][EVENT]167356 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [113792] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.038 [graph_manager.cc:3428][EVENT]167356 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.206 [graph_manager.cc:3467][EVENT]167356 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [145] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.241 [graph_manager.cc:3377][EVENT]167356 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [114475] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.257 [graph_manager.cc:1106][EVENT]167356 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [124011] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.270 [graph_manager.cc:1115][EVENT]167356 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:36.436.294 [graph_manager.cc:1130][EVENT]167356 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.327 [graph_manager.cc:1131][EVENT]167356 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.355 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.375 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.385 [graph_manager.cc:2837][EVENT]167356 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [41] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.518 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [24] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.436.531 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.436.541 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.436.550 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.436.559 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [7] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.436.567 [base_pass.cc:339][EVENT]167356 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [16] micro second, call num is [9] [INFO] GE(164046,python):2024-01-10-11:37:36.436.577 [graph_manager.cc:2864][EVENT]167356 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [175] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.590 [graph_manager.cc:2872][EVENT]167356 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.608 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.623 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.640 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.653 [compile_nodes_pass.cc:88][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.663 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.682 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.788 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [96] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.836 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.849 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.863 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.876 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.885 [graph_manager.cc:2927][EVENT]167356 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [280] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.898 [graph_manager.cc:2937][EVENT]167356 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.913 [graph_manager.cc:2943][EVENT]167356 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.436.924 [graph_manager.cc:2950][EVENT]167356 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.437.143 [graph_manager.cc:2958][EVENT]167356 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [51] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.437.176 [graph_manager.cc:1132][EVENT]167356 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [834] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.437.258 [graph_manager.cc:1135][EVENT]167356 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [69] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.437.300 [graph_manager.cc:2975][EVENT]167356 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [24] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.437.332 [graph_manager.cc:2981][EVENT]167356 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.437.346 [pass_manager.cc:82][EVENT]167356 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.437.356 [graph_manager.cc:2986][EVENT]167356 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.437.365 [graph_manager.cc:1136][EVENT]167356 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [91] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.437.741 [graph_manager.cc:3555][EVENT]167356 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [336] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.437.873 [engine_partitioner.cc:1139][EVENT]167356 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.437.912 [engine_partitioner.cc:1142][EVENT]167356 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.438.063 [engine_partitioner.cc:1148][EVENT]167356 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [140] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.438.099 [engine_partitioner.cc:1155][EVENT]167356 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [24] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.438.143 [engine_partitioner.cc:1164][EVENT]167356 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [33] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.438.168 [graph_builder.cc:865][EVENT]167356 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [357] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:36.438.676 [logger.cc:1071] 167356 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:37:36.438.710 [task_generator.cc:804][EVENT]167356 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [175] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.438.795 [task_generator.cc:805][EVENT]167356 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [72] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.440.477 [task_generator.cc:814][EVENT]167356 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1666] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.440.494 [task_generator.cc:954][EVENT]167356 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1959] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.440.565 [task_generator.cc:967][EVENT]167356 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [38] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:37:36.440.589 [logger.cc:1084] 167356 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:37:36.443.497 [graph_manager.cc:1152][EVENT]167356 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [6099] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.443.540 [graph_manager.cc:1164][EVENT]167356 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:37:36.443.578 [graph_manager.cc:1271][EVENT]167356 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [164178] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.443.590 [graph_manager.cc:1272][EVENT]167356 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:37:36.443.909 [atrace_api.c:93](tid:167356) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:37:36.443.929 [atrace_api.c:95](tid:167356) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:37:36.470.264 [graph_converter.cc:838][EVENT]167356 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [10711] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.470.477 [base_optimizer.cc:70][EVENT]167356 Run:[GEPERFTRACE] The time cost of ZeroCopy is [158] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.471.933 [base_optimizer.cc:70][EVENT]167356 Run:[GEPERFTRACE] The time cost of CEM is [1428] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.472.322 [copy_flow_launch_fuse.cc:395][EVENT]167356 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [358] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.472.350 [base_optimizer.cc:70][EVENT]167356 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [389] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.472.610 [base_optimizer.cc:70][EVENT]167356 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [246] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.472.710 [base_optimizer.cc:70][EVENT]167356 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [61] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.472.778 [base_optimizer.cc:70][EVENT]167356 Run:[GEPERFTRACE] The time cost of ZeroCopy is [50] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.473.226 [base_optimizer.cc:70][EVENT]167356 Run:[GEPERFTRACE] The time cost of CEM is [432] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.473.439 [copy_flow_launch_fuse.cc:395][EVENT]167356 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [186] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.473.463 [base_optimizer.cc:70][EVENT]167356 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [210] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.473.527 [base_optimizer.cc:70][EVENT]167356 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [53] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.473.583 [base_optimizer.cc:70][EVENT]167356 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [44] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.473.640 [base_optimizer.cc:70][EVENT]167356 Run:[GEPERFTRACE] The time cost of ZeroCopy is [44] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.473.889 [base_optimizer.cc:70][EVENT]167356 Run:[GEPERFTRACE] The time cost of CEM is [237] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.474.083 [copy_flow_launch_fuse.cc:395][EVENT]167356 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [175] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.474.101 [base_optimizer.cc:70][EVENT]167356 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [194] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.474.162 [base_optimizer.cc:70][EVENT]167356 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [51] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.474.218 [base_optimizer.cc:70][EVENT]167356 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [44] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.474.237 [graph_converter.cc:849][EVENT]167356 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [3925] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.474.928 [graph_converter.cc:853][EVENT]167356 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [680] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.476.828 [graph_converter.cc:857][EVENT]167356 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1877] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.477.208 [graph_converter.cc:862][EVENT]167356 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [349] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.577.256 [graph_var_manager.cc:1424][EVENT]167353 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:37:36.577.346 [graph_manager.cc:1248][EVENT]167353 PreRun:PreRun start: graph node size 5, session id 21, graph id 20, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:37:36.577.618 [atrace_api.c:28](tid:167353) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:37:36.577.646 [trace_rb_log.c:84](tid:167353) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:37:36.577.659 [atrace_api.c:32](tid:167353) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:37:36.577.677 [client_manager.cpp:157][SetProfilingCallback][tid:167353] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:37:36.578.085 [parallel_partitioner.cc:165][EVENT]167353 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.578.149 [parallel_partitioner.cc:178][EVENT]167353 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.578.196 [graph_prepare.cc:1378][EVENT]167353 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.578.381 [graph_manager.cc:1050][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [203] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.578.405 [graph_manager.cc:1052][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.578.552 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.578.583 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.578.630 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.578.644 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.578.689 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.578.701 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.578.718 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.578.819 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.578.839 [graph_manager.cc:1054][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [423] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.579.071 [graph_manager.cc:1055][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [217] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.580.234 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:36.580.261 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.580.273 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.580.283 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [396] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.580.292 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.580.301 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:36.580.311 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.580.319 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.580.339 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.582.608 [graph_manager.cc:1056][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3517] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.582.680 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.582.699 [graph_prepare.cc:1982][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [56] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.583.283 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:36.583.310 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.321 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.331 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [332] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.341 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.350 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:37:36.583.358 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.367 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.376 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.428 [graph_prepare.cc:1983][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [715] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.583.453 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.583.464 [graph_prepare.cc:1984][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.583.478 [graph_prepare.cc:1985][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.583.493 [graph_prepare.cc:1986][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.583.503 [graph_prepare.cc:1987][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.583.519 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.583.530 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.583.544 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.583.654 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.666 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.675 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.684 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.693 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.702 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [5] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.711 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.720 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.728 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.736 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.745 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [3] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.753 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.762 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.770 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.778 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.787 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:37:36.583.810 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.583.822 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.583.857 [graph_prepare.cc:1988][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [343] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.583.869 [graph_manager.cc:1065][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1227] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.596.002 [graph_manager.cc:1077][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12114] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.596.106 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.596.158 [graph_manager.cc:1080][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [120] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.604.607 [graph_manager.cc:1081][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [8422] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.604.650 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.604.667 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.604.679 [graph_manager.cc:1082][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.604.711 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.604.725 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.604.740 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.604.920 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [170] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.604.938 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.055 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [106] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.070 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.122 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [43] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.144 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.165 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.268 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [93] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.287 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.300 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.310 [graph_manager.cc:2700][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [605] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.587 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [5] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:37:36.605.602 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AddNPass is [3] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:37:36.605.612 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:37:36.605.621 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:37:36.605.641 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:37:36.605.650 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CastRemovePass is [52] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:37:36.605.659 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:37:36.605.668 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:37:36.605.676 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [12] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:37:36.605.685 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [4] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:37:36.605.740 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [18] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:37:36.605.748 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:37:36.605.757 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [22] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:37:36.605.765 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [10] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:37:36.605.774 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [7] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:37:36.605.784 [graph_manager.cc:2741][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [455] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.793 [graph_manager.cc:2752][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.817 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.829 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.853 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.868 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.880 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.893 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.913 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.927 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.940 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.957 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.971 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.605.983 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.606.006 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.606.020 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.606.029 [graph_manager.cc:2810][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [217] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.606.081 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:37:36.606.092 [graph_manager.cc:2821][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [54] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.606.119 [graph_manager.cc:1087][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1420] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.606.800 [graph_manager.cc:1088][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [666] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.606.865 [graph_manager.cc:1089][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.606.889 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.606.907 [graph_manager.cc:1097][EVENT]167353 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:37:36.606.930 [graph_manager.cc:3325][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.617.193 [engine_place.cc:144][EVENT]167353 Run:The time cost of AIcoreEngine::CheckSupported is [9991] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.617.224 [engine_place.cc:144][EVENT]167353 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.617.234 [engine_place.cc:144][EVENT]167353 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.617.333 [graph_manager.cc:3351][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10389] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.617.351 [graph_manager.cc:3364][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.617.436 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [32] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.617.471 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.617.674 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [192] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.617.770 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [33] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.617.820 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.617.857 [graph_manager.cc:3405][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [493] micro second. [INFO] GE(164046,python):2024-01-10-11:37:36.617.877 [graph_manager.cc:3412][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. - \ | / - [INFO] GE(164046,python):2024-01-10-11:38:24.184.621 [graph_manager.cc:3422][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [47566728] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.184.700 [graph_manager.cc:3428][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [26] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.184.958 [graph_manager.cc:3467][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [232] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.184.981 [graph_manager.cc:3377][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [47567618] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.001 [graph_manager.cc:1106][EVENT]167353 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [47578078] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.015 [graph_manager.cc:1115][EVENT]167353 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:38:24.185.048 [graph_manager.cc:1130][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.086 [graph_manager.cc:1131][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.123 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.148 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.160 [graph_manager.cc:2837][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [55] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.354 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [44] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:24.185.368 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:24.185.378 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [6] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:24.185.389 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of BitcastPass is [0] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:24.185.398 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [13] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:24.185.408 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [21] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:24.185.437 [graph_manager.cc:2864][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [256] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.451 [graph_manager.cc:2872][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.474 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.489 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.507 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.522 [compile_nodes_pass.cc:88][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.532 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.543 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.674 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [122] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.780 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [51] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.796 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.811 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.827 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.836 [graph_manager.cc:2927][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [368] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.850 [graph_manager.cc:2937][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.867 [graph_manager.cc:2943][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.185.879 [graph_manager.cc:2950][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.186.174 [graph_manager.cc:2958][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [66] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.186.210 [graph_manager.cc:1132][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [1110] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.186.318 [graph_manager.cc:1135][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [95] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.186.369 [graph_manager.cc:2975][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [24] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.186.401 [graph_manager.cc:2981][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.186.416 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.186.426 [graph_manager.cc:2986][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.186.435 [graph_manager.cc:1136][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [91] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.186.831 [graph_manager.cc:3555][EVENT]167353 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [347] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.186.992 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [43] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.187.032 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.187.225 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [181] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.187.267 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [27] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.187.315 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.187.343 [graph_builder.cc:865][EVENT]167353 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [430] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:38:24.187.810 [logger.cc:1071] 167353 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:38:24.187.844 [task_generator.cc:804][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [92] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.187.944 [task_generator.cc:805][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [88] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.190.073 [task_generator.cc:814][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [2113] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.190.091 [task_generator.cc:954][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2339] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.190.166 [task_generator.cc:967][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [41] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:38:24.190.190 [logger.cc:1084] 167353 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:38:24.191.382 [graph_manager.cc:1152][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4907] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.191.417 [graph_manager.cc:1164][EVENT]167353 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:38:24.191.459 [graph_manager.cc:1271][EVENT]167353 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [47613465] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.191.482 [graph_manager.cc:1272][EVENT]167353 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:38:24.191.812 [atrace_api.c:93](tid:167353) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:38:24.191.833 [atrace_api.c:95](tid:167353) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:38:24.221.760 [graph_converter.cc:838][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [11735] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.222.001 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [184] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.223.846 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [1818] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.224.343 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [465] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.224.371 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [495] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.224.660 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [276] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.224.761 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [76] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.224.843 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [61] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.225.414 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [553] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.225.677 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [234] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.225.730 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [287] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.225.812 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [71] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.225.885 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [59] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.225.958 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [60] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.226.231 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [259] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.226.469 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [220] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.226.490 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [242] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.226.567 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [67] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.226.638 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [57] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.226.658 [graph_converter.cc:849][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4849] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.227.545 [graph_converter.cc:853][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [875] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.229.911 [graph_converter.cc:857][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2342] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.230.405 [graph_converter.cc:862][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [449] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.330.159 [graph_var_manager.cc:1424][EVENT]167353 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:38:24.330.258 [graph_manager.cc:1248][EVENT]167353 PreRun:PreRun start: graph node size 4, session id 22, graph id 21, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:38:24.330.540 [atrace_api.c:28](tid:167353) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:38:24.330.567 [trace_rb_log.c:84](tid:167353) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:38:24.330.580 [atrace_api.c:32](tid:167353) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:38:24.330.606 [client_manager.cpp:157][SetProfilingCallback][tid:167353] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:38:24.331.007 [parallel_partitioner.cc:165][EVENT]167353 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.331.048 [parallel_partitioner.cc:178][EVENT]167353 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.331.093 [graph_prepare.cc:1378][EVENT]167353 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.331.231 [graph_manager.cc:1050][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [154] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.331.255 [graph_manager.cc:1052][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.331.394 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.331.424 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.331.473 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.331.486 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.331.531 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.331.545 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.331.563 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.331.660 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.331.679 [graph_manager.cc:1054][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [410] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.331.923 [graph_manager.cc:1055][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [231] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.332.973 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:38:24.333.001 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.333.014 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.333.023 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [340] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.333.033 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [14] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.333.042 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:38:24.333.051 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.333.059 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.333.068 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.335.266 [graph_manager.cc:1056][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3298] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.335.337 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.335.356 [graph_prepare.cc:1982][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [53] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.335.849 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:38:24.335.876 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.335.887 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.335.896 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [264] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.335.905 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.335.914 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:38:24.335.922 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.335.931 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.335.940 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.335.989 [graph_prepare.cc:1983][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [620] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.336.013 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.336.035 [graph_prepare.cc:1984][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.336.050 [graph_prepare.cc:1985][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.336.064 [graph_prepare.cc:1986][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.336.075 [graph_prepare.cc:1987][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.336.089 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.336.101 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.336.114 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.336.207 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.336.219 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.336.228 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.336.236 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.336.245 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.336.254 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.336.262 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.336.271 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.336.279 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.336.288 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.336.296 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.336.305 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.336.313 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.336.322 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.336.331 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.336.339 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.336.367 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.336.381 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.336.414 [graph_prepare.cc:1988][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [331] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.336.427 [graph_manager.cc:1065][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1126] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.348.943 [graph_manager.cc:1077][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12496] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.349.014 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.349.063 [graph_manager.cc:1080][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [85] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.354.406 [graph_manager.cc:1081][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [5327] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.354.448 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.354.462 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.354.474 [graph_manager.cc:1082][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [34] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.354.511 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.354.527 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.354.541 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.354.586 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.354.600 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.354.614 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.354.626 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.354.669 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [33] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.354.690 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.354.708 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.354.748 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.354.763 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.354.775 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.354.784 [graph_manager.cc:2700][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [283] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.354.899 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.354.912 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AddNPass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.354.922 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.354.931 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.354.940 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.354.949 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.354.957 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.354.966 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.354.974 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.354.983 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.354.991 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.355.000 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.355.008 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.355.016 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.355.025 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.355.034 [graph_manager.cc:2741][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [232] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.043 [graph_manager.cc:2752][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.065 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.076 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.102 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.116 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.128 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.140 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.159 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.173 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.186 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.196 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.209 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.221 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.240 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.252 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.261 [graph_manager.cc:2810][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [200] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.290 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.355.301 [graph_manager.cc:2821][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.327 [graph_manager.cc:1087][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [836] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.481 [graph_manager.cc:1088][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [141] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.523 [graph_manager.cc:1089][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.540 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.355.556 [graph_manager.cc:1097][EVENT]167353 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:38:24.355.576 [graph_manager.cc:3325][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.356.334 [engine_place.cc:144][EVENT]167353 Run:The time cost of AIcoreEngine::CheckSupported is [586] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.356.372 [engine_place.cc:144][EVENT]167353 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.356.382 [engine_place.cc:144][EVENT]167353 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.356.460 [graph_manager.cc:3351][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [869] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.356.478 [graph_manager.cc:3364][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.356.552 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.356.571 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.356.726 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [144] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.356.769 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [29] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.356.816 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.356.848 [graph_manager.cc:3405][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [357] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.356.866 [graph_manager.cc:3412][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.373.477 [graph_manager.cc:3422][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [16596] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.373.519 [graph_manager.cc:3428][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.373.659 [graph_manager.cc:3467][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [119] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.373.678 [graph_manager.cc:3377][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [17189] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.373.707 [graph_manager.cc:1106][EVENT]167353 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [18137] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.373.721 [graph_manager.cc:1115][EVENT]167353 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:38:24.373.745 [graph_manager.cc:1130][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.373.779 [graph_manager.cc:1131][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.373.804 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.373.832 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.373.842 [graph_manager.cc:2837][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [46] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.373.921 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.373.934 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.373.943 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.373.952 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.373.961 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.373.969 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.373.979 [graph_manager.cc:2864][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [120] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.373.991 [graph_manager.cc:2872][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.008 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.022 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.037 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.051 [compile_nodes_pass.cc:88][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.060 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.071 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.151 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [71] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.177 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.190 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.203 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.216 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.232 [graph_manager.cc:2927][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [227] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.245 [graph_manager.cc:2937][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.258 [graph_manager.cc:2943][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.269 [graph_manager.cc:2950][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.597 [graph_manager.cc:2958][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.629 [graph_manager.cc:1132][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [835] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.699 [graph_manager.cc:1135][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [57] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.731 [graph_manager.cc:2975][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.764 [graph_manager.cc:2981][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.779 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.790 [graph_manager.cc:2986][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.800 [graph_manager.cc:1136][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [84] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.374.916 [graph_manager.cc:3555][EVENT]167353 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [83] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.375.007 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.375.023 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.375.142 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [109] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.375.175 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.375.213 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [27] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.375.234 [graph_builder.cc:865][EVENT]167353 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [262] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:38:24.375.521 [logger.cc:1071] 167353 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:38:24.375.551 [task_generator.cc:804][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [72] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.375.623 [task_generator.cc:805][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [49] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.376.317 [task_generator.cc:814][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [679] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.376.332 [task_generator.cc:954][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [853] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.376.390 [task_generator.cc:967][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [31] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:38:24.376.408 [logger.cc:1084] 167353 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:38:24.377.066 [graph_manager.cc:1152][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2240] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.377.098 [graph_manager.cc:1164][EVENT]167353 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:38:24.377.133 [graph_manager.cc:1271][EVENT]167353 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [46219] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.377.145 [graph_manager.cc:1272][EVENT]167353 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:38:24.377.468 [atrace_api.c:93](tid:167353) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:38:24.377.486 [atrace_api.c:95](tid:167353) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:38:24.391.409 [graph_converter.cc:838][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [4625] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.391.579 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [122] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.392.088 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [486] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.392.300 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [187] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.392.321 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [210] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.392.537 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [204] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.392.577 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.392.609 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.392.807 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [186] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.392.892 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [66] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.392.906 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [81] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.392.936 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.392.961 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.392.987 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.393.077 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [68] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.393.147 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [58] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.393.159 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [70] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.393.185 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.393.209 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.393.222 [graph_converter.cc:849][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1770] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.393.447 [graph_converter.cc:853][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [215] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.394.191 [graph_converter.cc:857][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [729] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.394.339 [graph_converter.cc:862][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [121] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.489.620 [graph_var_manager.cc:1424][EVENT]167353 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:38:24.489.754 [graph_manager.cc:1248][EVENT]167353 PreRun:PreRun start: graph node size 4, session id 23, graph id 22, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:38:24.490.002 [atrace_api.c:28](tid:167353) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:38:24.490.029 [trace_rb_log.c:84](tid:167353) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:38:24.490.042 [atrace_api.c:32](tid:167353) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:38:24.490.061 [client_manager.cpp:157][SetProfilingCallback][tid:167353] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:38:24.490.451 [parallel_partitioner.cc:165][EVENT]167353 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.490.488 [parallel_partitioner.cc:178][EVENT]167353 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.490.531 [graph_prepare.cc:1378][EVENT]167353 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.490.774 [graph_manager.cc:1050][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [258] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.490.797 [graph_manager.cc:1052][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.490.935 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.490.964 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.491.012 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.491.045 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.491.093 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.491.106 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.491.122 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.491.224 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.491.243 [graph_manager.cc:1054][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [434] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.491.481 [graph_manager.cc:1055][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [223] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.492.451 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:38:24.492.480 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.492.492 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.492.502 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [308] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.492.512 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.492.521 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:38:24.492.530 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.492.539 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.492.547 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.494.586 [graph_manager.cc:1056][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3085] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.494.655 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.494.674 [graph_prepare.cc:1982][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [52] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.495.077 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:38:24.495.101 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.113 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.133 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [220] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.143 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.152 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:38:24.495.160 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.169 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [10] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.178 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.205 [graph_prepare.cc:1983][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [517] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.495.229 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.495.240 [graph_prepare.cc:1984][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.495.254 [graph_prepare.cc:1985][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.495.268 [graph_prepare.cc:1986][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.495.279 [graph_prepare.cc:1987][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.495.293 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.495.304 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.495.317 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.495.405 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.417 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.426 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.435 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.443 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.452 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.461 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.470 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.486 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of StopGradientPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.495 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.504 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.512 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.520 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.529 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.537 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.545 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.495.568 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.495.580 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.495.612 [graph_prepare.cc:1988][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [323] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.495.624 [graph_manager.cc:1065][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1004] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.509.572 [graph_manager.cc:1077][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13929] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.509.632 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.509.679 [graph_manager.cc:1080][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [71] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.089 [graph_manager.cc:1081][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [3382] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.130 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.144 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.155 [graph_manager.cc:1082][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [32] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.183 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.196 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.209 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.250 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.264 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.278 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.291 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.330 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [30] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.347 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.376 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.402 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.416 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.427 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.437 [graph_manager.cc:2700][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [258] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.551 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.513.564 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.513.573 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.513.582 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.513.591 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.513.599 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CastRemovePass is [10] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.513.608 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.513.616 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.513.624 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.513.633 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.513.641 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.513.657 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.513.666 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.513.675 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.513.683 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.513.704 [graph_manager.cc:2741][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [249] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.713 [graph_manager.cc:2752][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.735 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.747 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.764 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.779 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.790 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.801 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.820 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.833 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.846 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.856 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.875 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.887 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.906 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.920 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.930 [graph_manager.cc:2810][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [198] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.513.959 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.513.971 [graph_manager.cc:2821][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [32] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.514.004 [graph_manager.cc:1087][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [831] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.514.139 [graph_manager.cc:1088][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [123] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.514.179 [graph_manager.cc:1089][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.514.196 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.514.223 [graph_manager.cc:1097][EVENT]167353 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:38:24.514.245 [graph_manager.cc:3325][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.514.622 [engine_place.cc:144][EVENT]167353 Run:The time cost of AIcoreEngine::CheckSupported is [275] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.514.648 [engine_place.cc:144][EVENT]167353 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.514.658 [engine_place.cc:144][EVENT]167353 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.514.737 [graph_manager.cc:3351][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [478] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.514.755 [graph_manager.cc:3364][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.514.822 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.514.838 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.514.984 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [137] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.515.025 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.515.070 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [34] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.515.102 [graph_manager.cc:3405][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [334] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.515.119 [graph_manager.cc:3412][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.029 [graph_manager.cc:3422][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [163894] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.076 [graph_manager.cc:3428][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.219 [graph_manager.cc:3467][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [122] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.251 [graph_manager.cc:3377][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [164483] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.268 [graph_manager.cc:1106][EVENT]167353 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [165029] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.280 [graph_manager.cc:1115][EVENT]167353 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:38:24.679.304 [graph_manager.cc:1130][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.336 [graph_manager.cc:1131][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.360 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.376 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.386 [graph_manager.cc:2837][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.471 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.679.483 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.679.493 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.679.502 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.679.510 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.679.519 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:38:24.679.529 [graph_manager.cc:2864][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [125] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.540 [graph_manager.cc:2872][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.559 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.573 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.587 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.600 [compile_nodes_pass.cc:88][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.610 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.629 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.712 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [74] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.740 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.754 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.767 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.780 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.790 [graph_manager.cc:2927][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [234] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.802 [graph_manager.cc:2937][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.831 [graph_manager.cc:2943][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.679.846 [graph_manager.cc:2950][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.680.040 [graph_manager.cc:2958][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.680.071 [graph_manager.cc:1132][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [722] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.680.179 [graph_manager.cc:1135][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [94] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.680.215 [graph_manager.cc:2975][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.680.330 [graph_manager.cc:2981][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [102] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.680.346 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.680.357 [graph_manager.cc:2986][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.680.367 [graph_manager.cc:1136][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [172] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.680.482 [graph_manager.cc:3555][EVENT]167353 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [87] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.680.547 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.680.574 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.680.695 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [112] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.680.729 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.680.768 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.680.794 [graph_builder.cc:865][EVENT]167353 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [281] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.680.870 [graph_builder.cc:288][EVENT]167353 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [61] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.680.983 [graph_builder.cc:293][EVENT]167353 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [99] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.681.180 [model_builder.cc:1133][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [104] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.681.466 [block_mem_assigner.cc:4069][EVENT]169721 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164046,python):2024-01-10-11:38:24.681.467 [block_mem_assigner.cc:4069][EVENT]169722 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164046,python):2024-01-10-11:38:24.681.887 [graph_mem_assigner.cc:2166][EVENT]167353 SetInputOffset:[IMAS]AfterAssignMemory : online_22 memoffset[1024], memtype[2], theory_min[2048], zero_copy[1024], total_size[1024], no_reuse[1024], streams[1], topo_mode[DFS], mop[], io_reuse[0:0], alloc_mode[] [INFO] GE(164046,python):2024-01-10-11:38:24.681.983 [model_builder.cc:1144][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [782] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.682.010 [model_builder.cc:1152][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.682.026 [model_builder.cc:1157][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.682.145 [model_builder.cc:1167][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [107] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.682.164 [model_builder.cc:1174][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.682.184 [model_builder.cc:1180][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.682.218 [model_builder.cc:1184][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [24] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.682.236 [graph_builder.cc:304][EVENT]167353 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [1233] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:38:24.682.337 [logger.cc:1071] 167353 ModelBindStream: model_id=832, stream_id=65, flag=0. [INFO] GE(164046,python):2024-01-10-11:38:24.682.437 [task_generator.cc:804][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.682.514 [task_generator.cc:805][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [53] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.683.343 [task_generator.cc:814][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [815] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.683.356 [task_generator.cc:954][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [925] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.683.419 [task_generator.cc:967][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [35] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:38:24.683.439 [logger.cc:1084] 167353 ModelUnbindStream: model_id=832, stream_id=65, [INFO] GE(164046,python):2024-01-10-11:38:24.683.494 [graph_builder.cc:310][EVENT]167353 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [1246] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.683.615 [graph_manager.cc:1152][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3226] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.683.631 [graph_manager.cc:1164][EVENT]167353 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:38:24.683.662 [graph_manager.cc:1271][EVENT]167353 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [193296] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.683.673 [graph_manager.cc:1272][EVENT]167353 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:38:24.683.988 [atrace_api.c:93](tid:167353) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:38:24.684.004 [atrace_api.c:95](tid:167353) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:38:24.684.626 [model_introduction.cc:236][EVENT]167353 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(164046,python):2024-01-10-11:38:24.684.648 [model_introduction.cc:294][EVENT]167353 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(164046,python):2024-01-10-11:38:24.684.661 [model_introduction.cc:366][EVENT]167353 Data:model io_info size:222 [INFO] GE(164046,python):2024-01-10-11:38:24.779.573 [graph_var_manager.cc:1424][EVENT]167353 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:38:24.779.664 [graph_manager.cc:1248][EVENT]167353 PreRun:PreRun start: graph node size 5, session id 24, graph id 23, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:38:24.779.905 [atrace_api.c:28](tid:167353) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:38:24.779.936 [trace_rb_log.c:84](tid:167353) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:38:24.779.949 [atrace_api.c:32](tid:167353) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:38:24.779.967 [client_manager.cpp:157][SetProfilingCallback][tid:167353] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:38:24.780.385 [parallel_partitioner.cc:165][EVENT]167353 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.780.424 [parallel_partitioner.cc:178][EVENT]167353 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.780.469 [graph_prepare.cc:1378][EVENT]167353 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.780.638 [graph_manager.cc:1050][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [187] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.780.686 [graph_manager.cc:1052][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.780.849 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.780.878 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.780.926 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.780.939 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.780.985 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.780.999 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.781.017 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.781.119 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.781.139 [graph_manager.cc:1054][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [439] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.781.367 [graph_manager.cc:1055][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [215] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.782.774 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:24.782.803 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.782.814 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.782.824 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [463] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.782.834 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.782.842 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:24.782.851 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.782.860 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.782.869 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [7] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.786.952 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:24.786.983 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.786.995 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.787.015 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [364] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.787.026 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [12] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.787.035 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:24.787.044 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.787.054 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.787.063 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.788.421 [graph_manager.cc:1056][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [7033] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.788.489 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.788.508 [graph_prepare.cc:1982][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [55] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.789.111 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:24.789.136 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.147 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.157 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [342] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.167 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.175 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:24.789.184 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.193 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.202 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.255 [graph_prepare.cc:1983][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [733] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.789.278 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.789.290 [graph_prepare.cc:1984][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.789.304 [graph_prepare.cc:1985][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.789.318 [graph_prepare.cc:1986][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.789.340 [graph_prepare.cc:1987][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.789.356 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.789.368 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.789.382 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.789.482 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.494 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondPass is [4] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.503 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.513 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.521 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DropOutPass is [4] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.530 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.539 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.547 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.556 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.564 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.572 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.581 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.589 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.598 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.606 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.614 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:24.789.637 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.789.650 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.789.684 [graph_prepare.cc:1988][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [334] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.789.745 [graph_manager.cc:1065][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1293] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.802.435 [graph_manager.cc:1077][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12670] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.802.540 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.802.592 [graph_manager.cc:1080][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [122] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.808.235 [graph_manager.cc:1081][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [5628] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.808.276 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.808.292 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.808.303 [graph_manager.cc:1082][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.808.334 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.808.349 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.808.363 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.808.532 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [159] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.808.550 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.808.658 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [97] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.808.674 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.808.727 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [41] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.808.748 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.808.776 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.808.870 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [83] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.808.888 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.808.901 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.808.932 [graph_manager.cc:2700][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [604] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.809.194 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:24.809.209 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:24.809.219 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:24.809.229 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [4] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:24.809.237 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:24.809.246 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CastRemovePass is [44] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:24.809.255 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [4] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:24.809.264 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:24.809.272 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [11] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:24.809.281 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:24.809.290 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:24.809.298 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [16] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:24.809.307 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [23] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:24.809.315 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [11] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:24.809.323 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:24.809.333 [graph_manager.cc:2741][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [382] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.809.342 [graph_manager.cc:2752][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.809.365 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.809.377 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.809.401 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.809.415 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.809.427 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.809.450 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.809.470 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.809.485 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.809.497 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.809.508 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.809.523 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.809.534 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.809.558 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.809.571 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.809.580 [graph_manager.cc:2810][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [220] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.809.629 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:24.809.641 [graph_manager.cc:2821][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [52] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.809.669 [graph_manager.cc:1087][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1347] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.810.271 [graph_manager.cc:1088][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [587] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.810.336 [graph_manager.cc:1089][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [33] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.810.359 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.810.377 [graph_manager.cc:1097][EVENT]167353 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:38:24.810.399 [graph_manager.cc:3325][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.820.181 [engine_place.cc:144][EVENT]167353 Run:The time cost of AIcoreEngine::CheckSupported is [9516] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.820.212 [engine_place.cc:144][EVENT]167353 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.820.223 [engine_place.cc:144][EVENT]167353 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.820.325 [graph_manager.cc:3351][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [9911] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.820.358 [graph_manager.cc:3364][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.820.440 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.820.472 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.820.672 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [189] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.820.716 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.820.762 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.820.797 [graph_manager.cc:3405][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [427] micro second. [INFO] GE(164046,python):2024-01-10-11:38:24.820.816 [graph_manager.cc:3412][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.105.274 [graph_manager.cc:3422][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [284443] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.105.342 [graph_manager.cc:3428][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.105.585 [graph_manager.cc:3467][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [218] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.105.604 [graph_manager.cc:3377][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [285234] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.105.622 [graph_manager.cc:1106][EVENT]167353 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [295231] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.105.635 [graph_manager.cc:1115][EVENT]167353 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:38:25.105.663 [graph_manager.cc:1130][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.105.712 [graph_manager.cc:1131][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.105.747 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.105.768 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.105.779 [graph_manager.cc:2837][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [48] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.105.965 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [39] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:25.105.999 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:25.106.009 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [7] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:25.106.019 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of BitcastPass is [4] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:25.106.028 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [11] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:25.106.037 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [23] micro second, call num is [11] [INFO] GE(164046,python):2024-01-10-11:38:25.106.048 [graph_manager.cc:2864][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [249] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.061 [graph_manager.cc:2872][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.083 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.099 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.115 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.131 [compile_nodes_pass.cc:88][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.141 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.151 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.272 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [112] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.328 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [43] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.343 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.358 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.375 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.385 [graph_manager.cc:2927][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [307] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.397 [graph_manager.cc:2937][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.414 [graph_manager.cc:2943][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.433 [graph_manager.cc:2950][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.685 [graph_manager.cc:2958][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [64] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.720 [graph_manager.cc:1132][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [993] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.824 [graph_manager.cc:1135][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [90] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.865 [graph_manager.cc:2975][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [24] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.897 [graph_manager.cc:2981][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.911 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.922 [graph_manager.cc:2986][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.106.931 [graph_manager.cc:1136][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [90] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.107.297 [graph_manager.cc:3555][EVENT]167353 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [324] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.107.450 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [40] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.107.486 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.107.672 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [174] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.107.714 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [27] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.107.764 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.107.792 [graph_builder.cc:865][EVENT]167353 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [417] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:38:25.108.238 [logger.cc:1071] 167353 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:38:25.108.272 [task_generator.cc:804][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [98] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.108.364 [task_generator.cc:805][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [80] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.110.310 [task_generator.cc:814][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1931] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.110.328 [task_generator.cc:954][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2156] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.110.412 [task_generator.cc:967][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [40] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:38:25.110.438 [logger.cc:1084] 167353 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:38:25.158.036 [graph_manager.cc:1152][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [51069] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.158.120 [graph_manager.cc:1164][EVENT]167353 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:38:25.158.172 [graph_manager.cc:1271][EVENT]167353 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [377878] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.158.184 [graph_manager.cc:1272][EVENT]167353 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:38:25.158.529 [atrace_api.c:93](tid:167353) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:38:25.158.549 [atrace_api.c:95](tid:167353) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:38:25.351.868 [graph_converter.cc:838][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [66900] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.352.179 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [223] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.354.029 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [1821] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.354.526 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [464] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.354.553 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [494] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.354.839 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [272] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.354.933 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [69] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.355.015 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [61] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.355.581 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [549] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.355.830 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [221] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.355.854 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [246] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.355.928 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [63] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.355.998 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [55] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.356.067 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [55] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.356.335 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [255] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.356.561 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [206] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.356.582 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [228] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.356.685 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [64] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.356.755 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [55] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.356.776 [graph_converter.cc:849][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [4826] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.357.674 [graph_converter.cc:853][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [885] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.360.097 [graph_converter.cc:857][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2380] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.360.559 [graph_converter.cc:862][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [432] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.527.572 [graph_var_manager.cc:1424][EVENT]167352 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:38:25.527.675 [graph_manager.cc:1248][EVENT]167352 PreRun:PreRun start: graph node size 5, session id 25, graph id 24, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:38:25.528.165 [atrace_api.c:28](tid:167352) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:38:25.528.221 [trace_rb_log.c:84](tid:167352) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:38:25.528.234 [atrace_api.c:32](tid:167352) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:38:25.528.251 [client_manager.cpp:157][SetProfilingCallback][tid:167352] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:38:25.528.845 [parallel_partitioner.cc:165][EVENT]167352 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.528.885 [parallel_partitioner.cc:178][EVENT]167352 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.528.934 [graph_prepare.cc:1378][EVENT]167352 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.529.183 [graph_manager.cc:1050][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [268] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.529.209 [graph_manager.cc:1052][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.529.384 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.529.416 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.529.466 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [39] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.529.480 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.529.531 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.529.544 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.529.591 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.529.768 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [4] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.529.791 [graph_manager.cc:1054][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [570] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.530.029 [graph_manager.cc:1055][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [224] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.531.493 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.531.523 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.531.535 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [5] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.531.545 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [489] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.531.554 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [17] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.531.563 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.531.572 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.531.581 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.531.589 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [9] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.536.462 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.536.495 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.536.507 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.536.517 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [330] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.536.526 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.536.536 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.536.544 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.536.553 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [12] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.536.561 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.538.039 [graph_manager.cc:1056][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [7990] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.538.114 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.538.132 [graph_prepare.cc:1982][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [57] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.538.737 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.538.765 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.538.776 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.538.786 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [332] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.538.795 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [11] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.538.804 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.538.812 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.538.821 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.538.830 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.538.883 [graph_prepare.cc:1983][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [737] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.538.909 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.538.921 [graph_prepare.cc:1984][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.538.935 [graph_prepare.cc:1985][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.538.949 [graph_prepare.cc:1986][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.538.961 [graph_prepare.cc:1987][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.538.976 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.538.988 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.539.002 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.539.104 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.539.116 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.539.135 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.539.145 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.539.154 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.539.162 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.539.171 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [3] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.539.179 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.539.188 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.539.196 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.539.205 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.539.213 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.539.222 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.539.230 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.539.239 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.539.247 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.539.270 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.539.284 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.539.320 [graph_prepare.cc:1988][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [349] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.539.332 [graph_manager.cc:1065][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1257] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.552.596 [graph_manager.cc:1077][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13243] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.552.672 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.552.725 [graph_manager.cc:1080][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [92] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.558.260 [graph_manager.cc:1081][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [5519] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.558.316 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.558.332 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.558.344 [graph_manager.cc:1082][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.558.375 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.558.390 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.558.404 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.558.564 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [150] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.558.583 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.558.678 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [83] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.558.694 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.558.746 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [41] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.558.767 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.558.796 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.558.888 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [81] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.558.905 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.558.918 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.558.928 [graph_manager.cc:2700][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [559] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.559.177 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.559.192 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.559.202 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.559.211 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.559.220 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.559.251 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CastRemovePass is [43] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.559.260 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [4] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.559.268 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.559.277 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [12] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.559.285 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [4] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.559.294 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [18] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.559.303 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [13] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.559.311 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [21] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.559.320 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [9] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.559.328 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.559.337 [graph_manager.cc:2741][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [390] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.559.347 [graph_manager.cc:2752][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.559.372 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.559.384 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.559.407 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.559.422 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.559.435 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.559.449 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.559.470 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.559.485 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.559.498 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.559.508 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.559.521 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.559.539 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.559.564 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.559.578 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.559.587 [graph_manager.cc:2810][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [220] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.559.633 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.559.645 [graph_manager.cc:2821][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [50] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.559.674 [graph_manager.cc:1087][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1311] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.560.315 [graph_manager.cc:1088][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [627] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.560.382 [graph_manager.cc:1089][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.560.404 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.560.422 [graph_manager.cc:1097][EVENT]167352 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:38:25.560.446 [graph_manager.cc:3325][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.568.389 [engine_place.cc:144][EVENT]167352 Run:The time cost of AIcoreEngine::CheckSupported is [7701] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.568.422 [engine_place.cc:144][EVENT]167352 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.568.433 [engine_place.cc:144][EVENT]167352 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.568.538 [graph_manager.cc:3351][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [8078] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.568.556 [graph_manager.cc:3364][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.568.638 [engine_partitioner.cc:1139][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [30] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.568.670 [engine_partitioner.cc:1142][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.568.890 [engine_partitioner.cc:1148][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [210] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.568.939 [engine_partitioner.cc:1155][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [34] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.569.004 [engine_partitioner.cc:1164][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [43] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.569.045 [graph_manager.cc:3405][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [475] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.569.063 [graph_manager.cc:3412][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.675.297 [graph_manager.cc:3422][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [106218] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.675.353 [graph_manager.cc:3428][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.675.569 [graph_manager.cc:3467][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [194] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.675.589 [graph_manager.cc:3377][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [107020] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.675.607 [graph_manager.cc:1106][EVENT]167352 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [115169] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.675.620 [graph_manager.cc:1115][EVENT]167352 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:38:25.675.649 [graph_manager.cc:1130][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.675.682 [graph_manager.cc:1131][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.675.711 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.675.732 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.675.742 [graph_manager.cc:2837][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [43] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.675.897 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [33] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.675.911 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [5] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.675.921 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.675.930 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of BitcastPass is [3] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.675.939 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [9] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.675.948 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [16] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.675.958 [graph_manager.cc:2864][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [198] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.675.987 [graph_manager.cc:2872][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.007 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.022 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.039 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.054 [compile_nodes_pass.cc:88][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.064 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.075 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.196 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [111] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.252 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [43] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.266 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.281 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.296 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.305 [graph_manager.cc:2927][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [302] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.318 [graph_manager.cc:2937][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.333 [graph_manager.cc:2943][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.345 [graph_manager.cc:2950][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.602 [graph_manager.cc:2958][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [58] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.636 [graph_manager.cc:1132][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [940] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.726 [graph_manager.cc:1135][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [76] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.766 [graph_manager.cc:2975][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.804 [graph_manager.cc:2981][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.829 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.840 [graph_manager.cc:2986][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.676.849 [graph_manager.cc:1136][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [107] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.677.195 [graph_manager.cc:3555][EVENT]167352 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [303] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.677.337 [engine_partitioner.cc:1139][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.677.369 [engine_partitioner.cc:1142][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.677.569 [engine_partitioner.cc:1148][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [189] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.677.608 [engine_partitioner.cc:1155][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [25] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.677.655 [engine_partitioner.cc:1164][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.677.704 [graph_builder.cc:865][EVENT]167352 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [435] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:38:25.678.170 [logger.cc:1071] 167352 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:38:25.678.204 [task_generator.cc:804][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [108] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.678.300 [task_generator.cc:805][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [83] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.680.058 [task_generator.cc:814][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [1743] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.680.074 [task_generator.cc:954][EVENT]167352 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1978] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.680.148 [task_generator.cc:967][EVENT]167352 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [40] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:38:25.680.172 [logger.cc:1084] 167352 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:38:25.681.879 [graph_manager.cc:1152][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [4994] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.681.915 [graph_manager.cc:1164][EVENT]167352 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:38:25.681.954 [graph_manager.cc:1271][EVENT]167352 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [153207] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.681.964 [graph_manager.cc:1272][EVENT]167352 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:38:25.682.330 [atrace_api.c:93](tid:167352) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:38:25.682.359 [atrace_api.c:95](tid:167352) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:38:25.719.733 [graph_converter.cc:838][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [13275] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.719.981 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [200] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.722.201 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [2190] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.722.733 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [498] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.722.762 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [529] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.723.069 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [293] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.723.175 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [79] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.723.254 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [59] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.723.841 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [570] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.724.101 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [231] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.724.125 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [257] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.724.203 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [65] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.724.270 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [54] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.724.336 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [54] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.724.605 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [256] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.724.828 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [206] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.724.848 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [226] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.724.918 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [61] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.724.981 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [51] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.725.003 [graph_converter.cc:849][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [5228] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.725.898 [graph_converter.cc:853][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [884] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.728.233 [graph_converter.cc:857][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2307] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.728.708 [graph_converter.cc:862][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [443] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.845.149 [graph_var_manager.cc:1424][EVENT]167352 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:38:25.845.255 [graph_manager.cc:1248][EVENT]167352 PreRun:PreRun start: graph node size 7, session id 26, graph id 25, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:38:25.845.635 [atrace_api.c:28](tid:167352) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:38:25.845.672 [trace_rb_log.c:84](tid:167352) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:38:25.845.722 [atrace_api.c:32](tid:167352) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:38:25.845.743 [client_manager.cpp:157][SetProfilingCallback][tid:167352] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:38:25.846.249 [parallel_partitioner.cc:165][EVENT]167352 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [26] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.846.292 [parallel_partitioner.cc:178][EVENT]167352 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.846.345 [graph_prepare.cc:1378][EVENT]167352 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.846.612 [graph_manager.cc:1050][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [289] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.846.637 [graph_manager.cc:1052][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.846.828 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.846.859 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.846.908 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.846.921 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.846.975 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.846.989 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.847.007 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.847.129 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.847.150 [graph_manager.cc:1054][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [500] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.847.383 [graph_manager.cc:1055][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [219] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.848.995 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [5] micro second, call num is [14] [INFO] GE(164046,python):2024-01-10-11:38:25.849.026 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [6] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.849.063 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [5] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.849.074 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [560] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.849.084 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.849.093 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [5] micro second, call num is [14] [INFO] GE(164046,python):2024-01-10-11:38:25.849.102 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.849.111 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [21] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.849.120 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [8] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.850.912 [graph_manager.cc:1056][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3509] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.850.993 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.851.013 [graph_prepare.cc:1982][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [66] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.851.702 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [14] [INFO] GE(164046,python):2024-01-10-11:38:25.851.730 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.851.742 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.851.751 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [423] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.851.761 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [17] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.851.770 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [14] [INFO] GE(164046,python):2024-01-10-11:38:25.851.779 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [9] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.851.787 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.851.796 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.851.825 [graph_prepare.cc:1983][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [797] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.851.850 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.851.862 [graph_prepare.cc:1984][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.851.887 [graph_prepare.cc:1985][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.851.903 [graph_prepare.cc:1986][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.851.914 [graph_prepare.cc:1987][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.851.929 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.851.941 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.851.956 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.852.076 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.852.088 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.852.098 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.852.107 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.852.115 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DropOutPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.852.124 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.852.133 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.852.142 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.852.150 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.852.159 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.852.167 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.852.176 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.852.184 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [4] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.852.192 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.852.201 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [4] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.852.209 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.852.235 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.852.255 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.852.296 [graph_prepare.cc:1988][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [373] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.852.309 [graph_manager.cc:1065][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1362] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.866.083 [graph_manager.cc:1077][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13755] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.866.168 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.866.222 [graph_manager.cc:1080][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [103] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.870.619 [graph_manager.cc:1081][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [4380] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.870.663 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.870.678 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.870.690 [graph_manager.cc:1082][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.870.722 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.870.736 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.870.750 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.870.791 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [32] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.870.805 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.870.821 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.870.834 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.870.885 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [40] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.870.903 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.870.934 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.870.968 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [24] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.870.994 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.007 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.016 [graph_manager.cc:2700][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [299] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.174 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of EnterPass is [0] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.871.187 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.871.197 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.871.206 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [4] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.871.214 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.871.223 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CastRemovePass is [13] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.871.231 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.871.240 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [4] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.871.248 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.871.257 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.871.265 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.871.274 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.871.282 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [15] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.871.290 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.871.299 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [6] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.871.309 [graph_manager.cc:2741][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [273] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.318 [graph_manager.cc:2752][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.340 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.351 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.372 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.395 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.407 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.420 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.439 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.454 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.467 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.477 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.490 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.501 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.523 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.536 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.545 [graph_manager.cc:2810][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [209] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.582 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of IdentityPass is [0] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.871.593 [graph_manager.cc:2821][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [39] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.621 [graph_manager.cc:1087][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [911] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.762 [graph_manager.cc:1088][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [128] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.813 [graph_manager.cc:1089][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [30] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.831 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.871.875 [graph_manager.cc:1097][EVENT]167352 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:38:25.871.897 [graph_manager.cc:3325][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.872.430 [engine_place.cc:144][EVENT]167352 Run:The time cost of AIcoreEngine::CheckSupported is [394] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.872.458 [engine_place.cc:144][EVENT]167352 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.872.479 [engine_place.cc:144][EVENT]167352 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.872.578 [graph_manager.cc:3351][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [667] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.872.595 [graph_manager.cc:3364][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.872.668 [engine_partitioner.cc:1139][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [25] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.872.687 [engine_partitioner.cc:1142][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.872.977 [engine_partitioner.cc:1148][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [280] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.873.029 [engine_partitioner.cc:1155][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.873.084 [engine_partitioner.cc:1164][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [44] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.873.122 [graph_manager.cc:3405][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [515] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.873.141 [graph_manager.cc:3412][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.884.975 [graph_manager.cc:3422][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [11817] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.010 [graph_manager.cc:3428][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.170 [graph_manager.cc:3467][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [140] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.189 [graph_manager.cc:3377][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [12583] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.206 [graph_manager.cc:1106][EVENT]167352 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [13316] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.219 [graph_manager.cc:1115][EVENT]167352 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:38:25.885.242 [graph_manager.cc:1130][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.274 [graph_manager.cc:1131][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.299 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.317 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.336 [graph_manager.cc:2837][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [47] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.434 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.885.447 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.885.456 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.885.465 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of BitcastPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.885.474 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [10] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.885.482 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [8] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:38:25.885.492 [graph_manager.cc:2864][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [138] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.504 [graph_manager.cc:2872][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.522 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.536 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.551 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.564 [compile_nodes_pass.cc:88][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.574 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.584 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.711 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [117] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.745 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.758 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.772 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.784 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.794 [graph_manager.cc:2927][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [274] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.806 [graph_manager.cc:2937][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.828 [graph_manager.cc:2943][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.885.840 [graph_manager.cc:2950][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.886.065 [graph_manager.cc:2958][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [53] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.886.098 [graph_manager.cc:1132][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [810] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.886.170 [graph_manager.cc:1135][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [59] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.886.209 [graph_manager.cc:2975][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.886.243 [graph_manager.cc:2981][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.886.257 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.886.266 [graph_manager.cc:2986][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.886.275 [graph_manager.cc:1136][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [90] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.886.421 [graph_manager.cc:3555][EVENT]167352 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [109] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.886.530 [engine_partitioner.cc:1139][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.886.548 [engine_partitioner.cc:1142][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.886.779 [engine_partitioner.cc:1148][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [221] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.886.819 [engine_partitioner.cc:1155][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [25] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.886.865 [engine_partitioner.cc:1164][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [33] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.886.893 [graph_builder.cc:865][EVENT]167352 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [407] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:38:25.887.262 [logger.cc:1071] 167352 ModelBindStream: model_id=1344, stream_id=1601, flag=0. [INFO] GE(164046,python):2024-01-10-11:38:25.887.294 [task_generator.cc:804][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [95] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.887.376 [task_generator.cc:805][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [69] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.888.139 [task_generator.cc:814][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [748] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.888.164 [task_generator.cc:954][EVENT]167352 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [965] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.888.225 [task_generator.cc:967][EVENT]167352 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [34] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:38:25.888.243 [logger.cc:1084] 167352 ModelUnbindStream: model_id=1344, stream_id=1601, [INFO] GE(164046,python):2024-01-10-11:38:25.888.424 [graph_manager.cc:1152][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2118] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.888.442 [graph_manager.cc:1164][EVENT]167352 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:38:25.888.474 [graph_manager.cc:1271][EVENT]167352 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [42333] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.888.484 [graph_manager.cc:1272][EVENT]167352 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:38:25.888.833 [atrace_api.c:93](tid:167352) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:38:25.888.850 [atrace_api.c:95](tid:167352) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:38:25.896.393 [graph_converter.cc:838][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1788] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.896.485 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [45] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.897.159 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [654] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.897.466 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [281] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.897.490 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [306] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.897.550 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [50] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.897.587 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [24] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.897.620 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.897.882 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [251] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.897.997 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [95] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.898.013 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [111] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.898.052 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.898.081 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.898.113 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.898.226 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [104] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.898.317 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [79] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.898.338 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [101] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.898.374 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [27] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.898.403 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.898.418 [graph_converter.cc:849][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1984] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.898.744 [graph_converter.cc:853][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [316] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.899.909 [graph_converter.cc:857][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [1147] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.900.136 [graph_converter.cc:862][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [197] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.982.970 [graph_var_manager.cc:1424][EVENT]167354 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:38:25.983.069 [graph_manager.cc:1248][EVENT]167354 PreRun:PreRun start: graph node size 5, session id 27, graph id 26, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:38:25.983.347 [atrace_api.c:28](tid:167354) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:38:25.983.377 [trace_rb_log.c:84](tid:167354) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:38:25.983.390 [atrace_api.c:32](tid:167354) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:38:25.983.408 [client_manager.cpp:157][SetProfilingCallback][tid:167354] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:38:25.983.799 [parallel_partitioner.cc:165][EVENT]167354 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.983.836 [parallel_partitioner.cc:178][EVENT]167354 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.983.882 [graph_prepare.cc:1378][EVENT]167354 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.984.053 [graph_manager.cc:1050][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [187] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.984.076 [graph_manager.cc:1052][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.984.219 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.984.248 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.984.296 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.984.309 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.984.383 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.984.396 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.984.414 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.984.526 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.984.547 [graph_manager.cc:1054][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [459] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.984.791 [graph_manager.cc:1055][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [229] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.985.986 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.986.016 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.986.028 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.986.038 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferShapePass is [412] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.986.047 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.986.057 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.986.066 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.986.074 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [21] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.986.083 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferValuePass is [9] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.988.384 [graph_manager.cc:1056][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3574] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.988.454 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.988.472 [graph_prepare.cc:1982][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [55] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.989.052 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.989.078 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.089 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.099 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferShapePass is [328] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.108 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.128 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [10] [INFO] GE(164046,python):2024-01-10-11:38:25.989.138 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [7] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.147 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [13] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.156 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.209 [graph_prepare.cc:1983][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [723] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.989.234 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.989.246 [graph_prepare.cc:1984][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.989.261 [graph_prepare.cc:1985][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.989.275 [graph_prepare.cc:1986][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.989.286 [graph_prepare.cc:1987][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.989.301 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.989.313 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.989.327 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.989.427 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.440 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.449 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.458 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.467 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.476 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.484 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.493 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [3] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.501 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.510 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.526 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.535 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.544 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [4] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.553 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.561 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.570 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [5] [INFO] GE(164046,python):2024-01-10-11:38:25.989.592 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.989.605 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.989.638 [graph_prepare.cc:1988][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [341] micro second. [INFO] GE(164046,python):2024-01-10-11:38:25.989.651 [graph_manager.cc:1065][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1233] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.002.363 [graph_manager.cc:1077][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12693] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.002.437 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.002.486 [graph_manager.cc:1080][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [86] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.011.305 [graph_manager.cc:1081][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [8802] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.011.348 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.011.365 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.011.377 [graph_manager.cc:1082][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.011.409 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.011.424 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.011.439 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.011.624 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [175] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.011.653 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.011.776 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [111] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.011.792 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.011.847 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [44] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.011.869 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.011.889 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.011.992 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [92] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.010 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.023 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.033 [graph_manager.cc:2700][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [630] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.313 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:26.012.329 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:26.012.339 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:26.012.348 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:26.012.357 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:26.012.366 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CastRemovePass is [50] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:26.012.374 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:26.012.383 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [5] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:26.012.391 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [14] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:26.012.400 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [4] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:26.012.408 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [20] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:26.012.417 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [16] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:26.012.425 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [24] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:26.012.444 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [11] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:26.012.454 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [4] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:26.012.464 [graph_manager.cc:2741][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [412] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.473 [graph_manager.cc:2752][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.498 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.510 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.535 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.550 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.562 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.576 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.596 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.610 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.623 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.634 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.648 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.660 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.684 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.697 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.706 [graph_manager.cc:2810][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [214] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.756 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:38:26.012.768 [graph_manager.cc:2821][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [54] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.012.795 [graph_manager.cc:1087][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [1399] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.013.576 [graph_manager.cc:1088][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [747] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.013.643 [graph_manager.cc:1089][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.013.667 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.013.708 [graph_manager.cc:1097][EVENT]167354 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:38:26.013.735 [graph_manager.cc:3325][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.024.184 [engine_place.cc:144][EVENT]167354 Run:The time cost of AIcoreEngine::CheckSupported is [10170] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.024.216 [engine_place.cc:144][EVENT]167354 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.024.227 [engine_place.cc:144][EVENT]167354 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.024.325 [graph_manager.cc:3351][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [10577] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.024.344 [graph_manager.cc:3364][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.024.428 [engine_partitioner.cc:1139][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [32] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.024.462 [engine_partitioner.cc:1142][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.024.667 [engine_partitioner.cc:1148][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [195] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.024.714 [engine_partitioner.cc:1155][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [32] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.024.762 [engine_partitioner.cc:1164][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.024.798 [graph_manager.cc:3405][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [441] micro second. [INFO] GE(164046,python):2024-01-10-11:38:26.024.817 [graph_manager.cc:3412][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. \ | / - [INFO] GE(164046,python):2024-01-10-11:39:05.039.102 [graph_manager.cc:3422][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [39014269] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.039.172 [graph_manager.cc:3428][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.039.436 [graph_manager.cc:3467][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [235] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.039.458 [graph_manager.cc:3377][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [39015102] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.039.493 [graph_manager.cc:1106][EVENT]167354 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [39025766] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.039.507 [graph_manager.cc:1115][EVENT]167354 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:39:05.039.542 [graph_manager.cc:1130][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.039.577 [graph_manager.cc:1131][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.039.610 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.039.633 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.039.644 [graph_manager.cc:2837][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [49] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.039.832 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [40] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:39:05.039.847 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:39:05.039.857 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:39:05.039.866 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of BitcastPass is [3] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:39:05.039.875 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [10] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:39:05.039.884 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [20] micro second, call num is [12] [INFO] GE(164046,python):2024-01-10-11:39:05.039.895 [graph_manager.cc:2864][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [230] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.039.909 [graph_manager.cc:2872][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.039.930 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.039.946 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.039.964 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.039.979 [compile_nodes_pass.cc:88][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.039.989 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.039.999 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.040.139 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [122] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.040.196 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [44] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.040.211 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.040.225 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.040.241 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.040.250 [graph_manager.cc:2927][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [326] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.040.263 [graph_manager.cc:2937][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.040.280 [graph_manager.cc:2943][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.040.291 [graph_manager.cc:2950][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.040.590 [graph_manager.cc:2958][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [64] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.040.626 [graph_manager.cc:1132][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [1035] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.040.728 [graph_manager.cc:1135][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [90] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.040.773 [graph_manager.cc:2975][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [27] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.040.808 [graph_manager.cc:2981][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.040.823 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.040.833 [graph_manager.cc:2986][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.040.843 [graph_manager.cc:1136][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [97] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.041.212 [graph_manager.cc:3555][EVENT]167354 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [322] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.041.372 [engine_partitioner.cc:1139][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [41] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.041.411 [engine_partitioner.cc:1142][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.041.621 [engine_partitioner.cc:1148][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [189] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.041.664 [engine_partitioner.cc:1155][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.041.753 [engine_partitioner.cc:1164][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [78] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.041.784 [graph_builder.cc:865][EVENT]167354 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [488] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:39:05.042.323 [logger.cc:1071] 167354 ModelBindStream: model_id=1088, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:39:05.042.360 [task_generator.cc:804][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [155] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.042.462 [task_generator.cc:805][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [89] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.044.560 [task_generator.cc:814][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [2083] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.044.577 [task_generator.cc:954][EVENT]167354 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [2373] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.044.653 [task_generator.cc:967][EVENT]167354 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [41] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:39:05.044.678 [logger.cc:1084] 167354 ModelUnbindStream: model_id=1088, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:39:05.046.785 [graph_manager.cc:1152][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [5906] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.046.828 [graph_manager.cc:1164][EVENT]167354 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:39:05.046.870 [graph_manager.cc:1271][EVENT]167354 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [39063161] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.046.882 [graph_manager.cc:1272][EVENT]167354 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:39:05.047.289 [atrace_api.c:93](tid:167354) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:39:05.047.312 [atrace_api.c:95](tid:167354) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:39:05.076.204 [graph_converter.cc:838][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [11636] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.076.441 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [181] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.078.483 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [2014] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.078.999 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [483] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.079.028 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [513] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.079.325 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [284] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.079.428 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [77] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.079.525 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [63] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.080.102 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [561] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.080.365 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [233] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.080.389 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [260] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.080.467 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [66] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.080.537 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [56] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.080.607 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [58] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.080.875 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [254] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.081.111 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [217] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.081.131 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [237] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.081.205 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [64] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.081.274 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [56] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.081.294 [graph_converter.cc:849][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [5040] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.082.215 [graph_converter.cc:853][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [910] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.084.635 [graph_converter.cc:857][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [2391] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.085.115 [graph_converter.cc:862][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [450] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.179.141 [graph_var_manager.cc:1424][EVENT]167354 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:39:05.179.244 [graph_manager.cc:1248][EVENT]167354 PreRun:PreRun start: graph node size 4, session id 28, graph id 27, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:39:05.179.525 [atrace_api.c:28](tid:167354) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:39:05.179.552 [trace_rb_log.c:84](tid:167354) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:39:05.179.565 [atrace_api.c:32](tid:167354) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:39:05.179.589 [client_manager.cpp:157][SetProfilingCallback][tid:167354] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:39:05.179.974 [parallel_partitioner.cc:165][EVENT]167354 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.180.011 [parallel_partitioner.cc:178][EVENT]167354 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.180.082 [graph_prepare.cc:1378][EVENT]167354 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.180.448 [graph_manager.cc:1050][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [383] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.180.475 [graph_manager.cc:1052][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.180.615 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.180.645 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.180.693 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.180.706 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.180.752 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.180.765 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.180.781 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.180.886 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.180.906 [graph_manager.cc:1054][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [419] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.181.145 [graph_manager.cc:1055][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [225] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.182.263 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [4] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:39:05.182.294 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.182.306 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.182.315 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferShapePass is [333] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.182.325 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.182.334 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [4] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:39:05.182.343 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [15] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.182.351 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.182.360 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferValuePass is [9] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.184.481 [graph_manager.cc:1056][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3315] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.184.550 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.184.569 [graph_prepare.cc:1982][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [54] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.185.092 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:39:05.185.119 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.131 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.141 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferShapePass is [284] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.150 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.160 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [8] [INFO] GE(164046,python):2024-01-10-11:39:05.185.169 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.178 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.187 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.232 [graph_prepare.cc:1983][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [650] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.185.255 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.185.266 [graph_prepare.cc:1984][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.185.280 [graph_prepare.cc:1985][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.185.294 [graph_prepare.cc:1986][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.185.305 [graph_prepare.cc:1987][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.185.319 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.185.331 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.185.345 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.185.434 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.456 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.466 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrintOpPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.475 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.484 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.492 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.501 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.509 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.518 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of StopGradientPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.526 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.535 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.543 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.552 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.560 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.569 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.577 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.185.600 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.185.613 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.185.646 [graph_prepare.cc:1988][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [332] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.185.659 [graph_manager.cc:1065][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1144] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.198.573 [graph_manager.cc:1077][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12896] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.198.674 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.198.724 [graph_manager.cc:1080][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [115] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.207.701 [graph_manager.cc:1081][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [8962] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.207.756 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.207.771 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.207.784 [graph_manager.cc:1082][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.207.815 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.207.828 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.207.843 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.207.937 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [85] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.207.955 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.002 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.017 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.056 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.076 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.102 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.134 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.149 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.161 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.171 [graph_manager.cc:2700][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [361] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.291 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.208.304 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.208.314 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.208.323 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.208.332 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.208.350 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CastRemovePass is [9] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.208.360 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.208.368 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.208.377 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.208.385 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.208.394 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [8] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.208.402 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.208.411 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.208.419 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.208.428 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.208.438 [graph_manager.cc:2741][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [249] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.447 [graph_manager.cc:2752][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.469 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.481 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.499 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.514 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.526 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.539 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.557 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.571 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.583 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.593 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.605 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.622 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.641 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.653 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.662 [graph_manager.cc:2810][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [196] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.692 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.208.703 [graph_manager.cc:2821][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.730 [graph_manager.cc:1087][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [927] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.867 [graph_manager.cc:1088][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [125] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.908 [graph_manager.cc:1089][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.925 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.208.939 [graph_manager.cc:1097][EVENT]167354 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:39:05.208.959 [graph_manager.cc:3325][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.209.351 [engine_place.cc:144][EVENT]167354 Run:The time cost of AIcoreEngine::CheckSupported is [287] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.209.378 [engine_place.cc:144][EVENT]167354 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.209.387 [engine_place.cc:144][EVENT]167354 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.209.467 [graph_manager.cc:3351][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [494] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.209.484 [graph_manager.cc:3364][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.209.555 [engine_partitioner.cc:1139][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.209.573 [engine_partitioner.cc:1142][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.209.751 [engine_partitioner.cc:1148][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [168] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.209.798 [engine_partitioner.cc:1155][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.209.855 [engine_partitioner.cc:1164][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.209.888 [graph_manager.cc:3405][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [391] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.209.906 [graph_manager.cc:3412][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.169 [graph_manager.cc:3422][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [17249] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.208 [graph_manager.cc:3428][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.341 [graph_manager.cc:3467][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [114] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.360 [graph_manager.cc:3377][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [17863] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.376 [graph_manager.cc:1106][EVENT]167354 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [18422] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.388 [graph_manager.cc:1115][EVENT]167354 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:39:05.227.410 [graph_manager.cc:1130][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.442 [graph_manager.cc:1131][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.465 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.482 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.491 [graph_manager.cc:2837][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.573 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [16] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.227.585 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.227.594 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.227.603 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.227.613 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [6] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.227.621 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [9] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:39:05.227.631 [graph_manager.cc:2864][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [124] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.653 [graph_manager.cc:2872][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.672 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.686 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.701 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.715 [compile_nodes_pass.cc:88][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.724 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.735 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.813 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [69] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.840 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.853 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.866 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.879 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.888 [graph_manager.cc:2927][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [219] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.900 [graph_manager.cc:2937][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.913 [graph_manager.cc:2943][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.227.924 [graph_manager.cc:2950][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.228.112 [graph_manager.cc:2958][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.228.144 [graph_manager.cc:1132][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [689] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.228.213 [graph_manager.cc:1135][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [55] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.228.244 [graph_manager.cc:2975][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.228.276 [graph_manager.cc:2981][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.228.301 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.228.312 [graph_manager.cc:2986][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.228.321 [graph_manager.cc:1136][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [92] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.228.457 [graph_manager.cc:3555][EVENT]167354 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [105] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.228.548 [engine_partitioner.cc:1139][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.228.565 [engine_partitioner.cc:1142][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.228.682 [engine_partitioner.cc:1148][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [107] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.228.715 [engine_partitioner.cc:1155][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.228.753 [engine_partitioner.cc:1164][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [26] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.228.776 [graph_builder.cc:865][EVENT]167354 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [263] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:39:05.229.155 [logger.cc:1071] 167354 ModelBindStream: model_id=1856, stream_id=65, flag=0. [INFO] GE(164046,python):2024-01-10-11:39:05.229.188 [task_generator.cc:804][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [128] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.229.249 [task_generator.cc:805][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [49] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.230.128 [task_generator.cc:814][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [864] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.230.144 [task_generator.cc:954][EVENT]167354 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1085] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.230.202 [task_generator.cc:967][EVENT]167354 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [31] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:39:05.230.221 [logger.cc:1084] 167354 ModelUnbindStream: model_id=1856, stream_id=65, [INFO] GE(164046,python):2024-01-10-11:39:05.231.385 [graph_manager.cc:1152][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [3038] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.231.417 [graph_manager.cc:1164][EVENT]167354 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:39:05.231.451 [graph_manager.cc:1271][EVENT]167354 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [51568] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.231.462 [graph_manager.cc:1272][EVENT]167354 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:39:05.231.778 [atrace_api.c:93](tid:167354) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:39:05.231.807 [atrace_api.c:95](tid:167354) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:39:05.243.152 [graph_converter.cc:838][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [3682] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.243.318 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [118] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.243.806 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [464] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.244.014 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [182] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.244.035 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [206] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.244.254 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [206] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.244.295 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.244.326 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.244.523 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [185] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.244.607 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [66] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.244.622 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [81] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.244.651 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.244.676 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.244.702 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.244.778 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [66] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.244.845 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [56] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.244.857 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [68] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.244.883 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.244.907 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.244.921 [graph_converter.cc:849][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1729] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.245.144 [graph_converter.cc:853][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [213] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.245.863 [graph_converter.cc:857][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [704] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.246.014 [graph_converter.cc:862][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [123] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.341.229 [graph_var_manager.cc:1424][EVENT]167353 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:39:05.341.324 [graph_manager.cc:1248][EVENT]167353 PreRun:PreRun start: graph node size 7, session id 29, graph id 28, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:39:05.341.978 [atrace_api.c:28](tid:167353) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:39:05.342.038 [trace_rb_log.c:84](tid:167353) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:39:05.342.051 [atrace_api.c:32](tid:167353) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:39:05.342.068 [client_manager.cpp:157][SetProfilingCallback][tid:167353] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:39:05.342.775 [parallel_partitioner.cc:165][EVENT]167353 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [25] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.342.819 [parallel_partitioner.cc:178][EVENT]167353 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.342.869 [graph_prepare.cc:1378][EVENT]167353 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.343.343 [graph_manager.cc:1050][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [495] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.343.373 [graph_manager.cc:1052][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.343.560 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.343.590 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.343.640 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.343.653 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.343.703 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.343.716 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.343.734 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.343.845 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.343.866 [graph_manager.cc:1054][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [480] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.344.094 [graph_manager.cc:1055][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [216] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.345.542 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [14] [INFO] GE(164046,python):2024-01-10-11:39:05.345.572 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.345.610 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.345.621 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [501] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.345.630 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [20] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.345.639 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [14] [INFO] GE(164046,python):2024-01-10-11:39:05.345.648 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [12] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.345.657 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.345.665 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.347.296 [graph_manager.cc:1056][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3182] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.347.376 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.347.394 [graph_prepare.cc:1982][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [63] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.348.102 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [14] [INFO] GE(164046,python):2024-01-10-11:39:05.348.130 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.141 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [4] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.151 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [437] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.160 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [16] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.169 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [14] [INFO] GE(164046,python):2024-01-10-11:39:05.348.178 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [10] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.187 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [18] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.195 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [5] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.224 [graph_prepare.cc:1983][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [816] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.348.247 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.348.258 [graph_prepare.cc:1984][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.348.283 [graph_prepare.cc:1985][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.348.299 [graph_prepare.cc:1986][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.348.310 [graph_prepare.cc:1987][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.348.325 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.348.337 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.348.351 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.348.468 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.481 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.491 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.500 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.509 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DropOutPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.517 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.526 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.535 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.543 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.551 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.559 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.568 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.576 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.584 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [6] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.593 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.601 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.348.626 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.348.647 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.348.687 [graph_prepare.cc:1988][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [368] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.348.699 [graph_manager.cc:1065][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1368] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.361.944 [graph_manager.cc:1077][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13225] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.362.026 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.362.082 [graph_manager.cc:1080][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [101] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.366.312 [graph_manager.cc:1081][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [4212] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.366.354 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.366.369 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.366.382 [graph_manager.cc:1082][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.366.414 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.366.428 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.366.441 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.366.474 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.366.489 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.366.505 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.366.517 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.366.566 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [39] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.366.585 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.366.615 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.366.650 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [24] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.366.675 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.366.688 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.366.697 [graph_manager.cc:2700][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [289] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.366.851 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.366.866 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AddNPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.366.876 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.366.885 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.366.894 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.366.902 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CastRemovePass is [12] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.366.911 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.366.920 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.366.928 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.366.937 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.366.945 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [10] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.366.953 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.366.962 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.366.970 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.366.978 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.366.988 [graph_manager.cc:2741][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [273] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.366.997 [graph_manager.cc:2752][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.020 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.031 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.051 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.074 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.086 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.099 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.119 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.133 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.146 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.157 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.170 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.182 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.203 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.216 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.225 [graph_manager.cc:2810][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [211] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.264 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.367.276 [graph_manager.cc:2821][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [41] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.303 [graph_manager.cc:1087][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [902] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.443 [graph_manager.cc:1088][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [126] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.491 [graph_manager.cc:1089][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.508 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.367.550 [graph_manager.cc:1097][EVENT]167353 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:39:05.367.575 [graph_manager.cc:3325][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.368.087 [engine_place.cc:144][EVENT]167353 Run:The time cost of AIcoreEngine::CheckSupported is [401] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.368.115 [engine_place.cc:144][EVENT]167353 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [12] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.368.125 [engine_place.cc:144][EVENT]167353 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.368.226 [graph_manager.cc:3351][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [637] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.368.244 [graph_manager.cc:3364][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.368.314 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [24] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.368.332 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.368.587 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [245] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.368.638 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.368.690 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [40] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.368.727 [graph_manager.cc:3405][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [472] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.368.745 [graph_manager.cc:3412][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.379.897 [graph_manager.cc:3422][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [11136] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.379.933 [graph_manager.cc:3428][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.083 [graph_manager.cc:3467][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [129] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.101 [graph_manager.cc:3377][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [11846] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.117 [graph_manager.cc:1106][EVENT]167353 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [12549] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.129 [graph_manager.cc:1115][EVENT]167353 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:39:05.380.152 [graph_manager.cc:1130][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.183 [graph_manager.cc:1131][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.208 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.225 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.245 [graph_manager.cc:2837][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [47] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.343 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [20] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.380.356 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.380.366 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.380.375 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.380.384 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [7] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.380.392 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [4] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.380.402 [graph_manager.cc:2864][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [138] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.414 [graph_manager.cc:2872][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.431 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.444 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.459 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.473 [compile_nodes_pass.cc:88][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.482 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.492 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.601 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [99] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.630 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.642 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.656 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.669 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.678 [graph_manager.cc:2927][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [250] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.690 [graph_manager.cc:2937][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.713 [graph_manager.cc:2943][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.724 [graph_manager.cc:2950][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.923 [graph_manager.cc:2958][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [48] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.380.955 [graph_manager.cc:1132][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [758] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.381.026 [graph_manager.cc:1135][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [58] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.381.067 [graph_manager.cc:2975][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [26] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.381.101 [graph_manager.cc:2981][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.381.115 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.381.125 [graph_manager.cc:2986][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.381.134 [graph_manager.cc:1136][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [93] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.381.277 [graph_manager.cc:3555][EVENT]167353 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [105] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.381.388 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.381.407 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.381.608 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [191] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.381.646 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [24] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.381.705 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [47] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.381.733 [graph_builder.cc:865][EVENT]167353 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [391] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:39:05.382.139 [logger.cc:1071] 167353 ModelBindStream: model_id=64, stream_id=321, flag=0. [INFO] GE(164046,python):2024-01-10-11:39:05.382.172 [task_generator.cc:804][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [129] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.382.249 [task_generator.cc:805][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [64] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.383.166 [task_generator.cc:814][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [902] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.383.192 [task_generator.cc:954][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [1150] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.383.250 [task_generator.cc:967][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [32] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:39:05.383.269 [logger.cc:1084] 167353 ModelUnbindStream: model_id=64, stream_id=321, [INFO] GE(164046,python):2024-01-10-11:39:05.383.443 [graph_manager.cc:1152][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2278] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.383.463 [graph_manager.cc:1164][EVENT]167353 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:39:05.383.494 [graph_manager.cc:1271][EVENT]167353 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [40824] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.383.507 [graph_manager.cc:1272][EVENT]167353 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:39:05.383.818 [atrace_api.c:93](tid:167353) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:39:05.383.835 [atrace_api.c:95](tid:167353) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:39:05.391.065 [graph_converter.cc:838][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1708] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.391.148 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.391.762 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [596] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.392.034 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [245] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.392.058 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [271] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.392.112 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [43] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.392.148 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.392.181 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.392.390 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [198] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.392.494 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [85] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.392.510 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [103] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.392.549 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.392.579 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.392.610 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.392.718 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [98] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.392.809 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [79] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.392.831 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [102] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.392.868 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [27] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.392.897 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.392.910 [graph_converter.cc:849][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1805] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.393.212 [graph_converter.cc:853][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [292] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.394.227 [graph_converter.cc:857][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [999] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.394.435 [graph_converter.cc:862][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [178] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.477.574 [graph_var_manager.cc:1424][EVENT]167354 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:39:05.477.670 [graph_manager.cc:1248][EVENT]167354 PreRun:PreRun start: graph node size 7, session id 30, graph id 29, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:39:05.477.964 [atrace_api.c:28](tid:167354) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:39:05.477.992 [trace_rb_log.c:84](tid:167354) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:39:05.478.004 [atrace_api.c:32](tid:167354) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:39:05.478.022 [client_manager.cpp:157][SetProfilingCallback][tid:167354] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:39:05.478.431 [parallel_partitioner.cc:165][EVENT]167354 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.478.469 [parallel_partitioner.cc:178][EVENT]167354 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.478.518 [graph_prepare.cc:1378][EVENT]167354 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.478.708 [graph_manager.cc:1050][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [209] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.478.732 [graph_manager.cc:1052][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.478.906 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.478.936 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.478.984 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.478.997 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.479.066 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.479.080 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.479.098 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.479.206 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.479.226 [graph_manager.cc:1054][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [480] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.479.456 [graph_manager.cc:1055][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [216] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.480.852 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [3] micro second, call num is [14] [INFO] GE(164046,python):2024-01-10-11:39:05.480.882 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [6] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.480.894 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.480.904 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferShapePass is [490] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.480.914 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [20] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.480.923 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [3] micro second, call num is [14] [INFO] GE(164046,python):2024-01-10-11:39:05.480.931 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [14] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.480.940 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [23] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.480.949 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferValuePass is [8] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.482.585 [graph_manager.cc:1056][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [3110] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.482.662 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondRemovePass is [4] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.482.681 [graph_prepare.cc:1982][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [61] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.483.405 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [5] micro second, call num is [14] [INFO] GE(164046,python):2024-01-10-11:39:05.483.431 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.442 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.452 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferShapePass is [433] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.461 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.481 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [5] micro second, call num is [14] [INFO] GE(164046,python):2024-01-10-11:39:05.483.491 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [9] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.500 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [17] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.508 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of InferValuePass is [6] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.567 [graph_prepare.cc:1983][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [872] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.483.590 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.483.601 [graph_prepare.cc:1984][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.483.615 [graph_prepare.cc:1985][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.483.630 [graph_prepare.cc:1986][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.483.641 [graph_prepare.cc:1987][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.483.656 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.483.668 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.483.682 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.483.799 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of EnterPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.814 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.823 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrintOpPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.832 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [4] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.841 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DropOutPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.849 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.858 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.866 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.875 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of StopGradientPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.883 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.901 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.910 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.919 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.927 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [8] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.936 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.945 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of IdentityPass is [5] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.483.970 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.483.983 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.484.019 [graph_prepare.cc:1988][EVENT]167354 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [369] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.484.032 [graph_manager.cc:1065][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [1413] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.497.062 [graph_manager.cc:1077][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13010] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.497.191 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.497.247 [graph_manager.cc:1080][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [149] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.501.373 [graph_manager.cc:1081][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [4108] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.501.415 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.501.431 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.501.442 [graph_manager.cc:1082][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.501.473 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.501.487 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.501.501 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.501.533 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.501.557 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.501.574 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.501.587 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.501.634 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.501.653 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.501.683 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.501.728 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.501.743 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.501.756 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.501.765 [graph_manager.cc:2700][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [296] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.501.918 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.501.932 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AddNPass is [0] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.501.942 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [5] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.501.951 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.501.960 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of MergePass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.501.969 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CastRemovePass is [13] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.501.977 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.501.986 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.501.994 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.502.002 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [4] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.502.011 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.502.019 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [5] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.502.028 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [14] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.502.045 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.502.054 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.502.064 [graph_manager.cc:2741][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [281] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.073 [graph_manager.cc:2752][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.094 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.105 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.125 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.141 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.153 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.167 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.187 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.202 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.215 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.225 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.239 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.250 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.270 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.284 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.294 [graph_manager.cc:2810][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [204] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.331 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of IdentityPass is [3] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.502.342 [graph_manager.cc:2821][EVENT]167354 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [40] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.369 [graph_manager.cc:1087][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [908] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.514 [graph_manager.cc:1088][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [126] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.561 [graph_manager.cc:1089][EVENT]167354 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [27] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.579 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.502.621 [graph_manager.cc:1097][EVENT]167354 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:39:05.502.644 [graph_manager.cc:3325][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.503.126 [engine_place.cc:144][EVENT]167354 Run:The time cost of AIcoreEngine::CheckSupported is [373] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.503.153 [engine_place.cc:144][EVENT]167354 Run:The time cost of DNN_VM_GE_LOCAL_OP_STORE::CheckSupported is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.503.163 [engine_place.cc:144][EVENT]167354 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.503.255 [graph_manager.cc:3351][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [598] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.503.273 [graph_manager.cc:3364][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.503.342 [engine_partitioner.cc:1139][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [24] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.503.360 [engine_partitioner.cc:1142][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.503.598 [engine_partitioner.cc:1148][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [228] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.503.649 [engine_partitioner.cc:1155][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [36] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.503.697 [engine_partitioner.cc:1164][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.503.731 [graph_manager.cc:3405][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [445] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.503.748 [graph_manager.cc:3412][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.516.327 [graph_manager.cc:3422][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [12565] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.516.364 [graph_manager.cc:3428][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.516.514 [graph_manager.cc:3467][EVENT]167354 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [131] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.516.533 [graph_manager.cc:3377][EVENT]167354 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [13249] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.516.558 [graph_manager.cc:1106][EVENT]167354 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [13920] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.516.571 [graph_manager.cc:1115][EVENT]167354 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:39:05.516.594 [graph_manager.cc:1130][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.516.626 [graph_manager.cc:1131][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.516.650 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.516.668 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.516.678 [graph_manager.cc:2837][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [38] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.516.773 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [19] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.516.786 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.516.795 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of CondRemovePass is [2] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.516.804 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.516.813 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [7] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.516.822 [base_pass.cc:339][EVENT]167354 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [5] micro second, call num is [7] [INFO] GE(164046,python):2024-01-10-11:39:05.516.831 [graph_manager.cc:2864][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [136] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.516.844 [graph_manager.cc:2872][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.516.862 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.516.875 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.516.891 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.516.905 [compile_nodes_pass.cc:88][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.516.915 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.516.924 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.517.040 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [99] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.517.068 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.517.082 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.517.095 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.517.108 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.517.117 [graph_manager.cc:2927][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [258] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.517.129 [graph_manager.cc:2937][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.517.144 [graph_manager.cc:2943][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.517.156 [graph_manager.cc:2950][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.517.348 [graph_manager.cc:2958][EVENT]167354 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [48] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.517.381 [graph_manager.cc:1132][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [742] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.517.451 [graph_manager.cc:1135][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [57] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.517.488 [graph_manager.cc:2975][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.517.519 [graph_manager.cc:2981][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.517.533 [pass_manager.cc:82][EVENT]167354 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.517.543 [graph_manager.cc:2986][EVENT]167354 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.517.553 [graph_manager.cc:1136][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [87] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.517.734 [graph_manager.cc:3555][EVENT]167354 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [146] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.517.841 [engine_partitioner.cc:1139][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.517.859 [engine_partitioner.cc:1142][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.518.041 [engine_partitioner.cc:1148][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [172] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.518.091 [engine_partitioner.cc:1155][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [25] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.518.132 [engine_partitioner.cc:1164][EVENT]167354 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [30] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.518.156 [graph_builder.cc:865][EVENT]167354 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [358] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:39:05.518.487 [logger.cc:1071] 167354 ModelBindStream: model_id=1344, stream_id=1601, flag=0. [INFO] GE(164046,python):2024-01-10-11:39:05.518.519 [task_generator.cc:804][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [82] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.518.592 [task_generator.cc:805][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [62] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.519.269 [task_generator.cc:814][EVENT]167354 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [662] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.519.284 [task_generator.cc:954][EVENT]167354 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [849] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.519.343 [task_generator.cc:967][EVENT]167354 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [33] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:39:05.519.361 [logger.cc:1084] 167354 ModelUnbindStream: model_id=1344, stream_id=1601, [INFO] GE(164046,python):2024-01-10-11:39:05.519.533 [graph_manager.cc:1152][EVENT]167354 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [1952] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.519.553 [graph_manager.cc:1164][EVENT]167354 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:39:05.519.584 [graph_manager.cc:1271][EVENT]167354 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [41249] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.519.596 [graph_manager.cc:1272][EVENT]167354 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:39:05.519.903 [atrace_api.c:93](tid:167354) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:39:05.519.919 [atrace_api.c:95](tid:167354) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:39:05.527.094 [graph_converter.cc:838][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1705] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.527.177 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.527.788 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [593] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.528.058 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [244] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.528.080 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [267] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.528.133 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [41] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.528.166 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.528.200 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.528.418 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [196] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.528.523 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [84] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.528.538 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [100] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.528.575 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.528.606 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.528.638 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.528.745 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CEM is [97] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.528.834 [copy_flow_launch_fuse.cc:395][EVENT]167354 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [78] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.528.846 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [90] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.528.880 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [26] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.528.909 [base_optimizer.cc:70][EVENT]167354 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.528.923 [graph_converter.cc:849][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [1788] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.529.219 [graph_converter.cc:853][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [286] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.530.236 [graph_converter.cc:857][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [999] micro second. [INFO] GE(164046,python):2024-01-10-11:39:05.530.443 [graph_converter.cc:862][EVENT]167354 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [177] micro second. [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:39:05.534.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 1, total running count: 1 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:27.366.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:27.369.676 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 102 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:27.393.217 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 2, total running count: 2 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:27.779.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:27.782.031 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 103 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:27.795.068 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 3, total running count: 3 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:27.795.478 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:27.797.838 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 104 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:27.810.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 4, total running count: 4 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:27.810.620 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:27.813.414 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 105 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:27.825.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 5, total running count: 5 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:27.826.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:27.827.893 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 106 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:27.840.641 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 6, total running count: 6 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:27.841.485 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:27.843.716 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 107 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:27.856.115 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 7, total running count: 7 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:27.856.724 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:27.859.514 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 108 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:27.871.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 8, total running count: 8 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:27.872.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:27.873.881 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 109 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:27.886.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 9, total running count: 9 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:27.887.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:27.889.836 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 110 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:27.901.742 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 10, total running count: 10 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:27.902.746 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:27.905.671 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 111 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:27.916.885 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 11, total running count: 11 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:27.917.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:27.920.129 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 112 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:27.932.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 12, total running count: 12 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:27.932.642 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:27.934.750 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 113 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:27.946.759 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 13, total running count: 13 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:27.947.697 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:27.949.391 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 114 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:27.961.612 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 14, total running count: 14 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:27.962.617 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:27.965.268 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 115 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:27.976.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 15, total running count: 15 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:27.977.618 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:27.979.592 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 116 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:27.991.455 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 16, total running count: 16 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:27.992.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:27.995.052 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 117 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.006.649 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 17, total running count: 17 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.007.627 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.009.346 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 118 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.021.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 18, total running count: 18 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.022.583 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.024.721 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 119 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:28.036.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 19, total running count: 19 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.037.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.040.144 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 120 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.051.478 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 20, total running count: 20 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.052.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.054.666 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 121 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.066.424 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 21, total running count: 21 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.067.379 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.069.462 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 122 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.081.182 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 22, total running count: 22 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.082.586 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.084.157 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 123 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.096.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 23, total running count: 23 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.097.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.099.896 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 124 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.111.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 24, total running count: 24 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.112.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.114.744 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 125 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.126.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 25, total running count: 25 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.127.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.129.282 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 126 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.141.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 26, total running count: 26 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.142.077 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.144.852 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 127 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.155.993 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 27, total running count: 27 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.156.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.159.209 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 128 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.170.781 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 28, total running count: 28 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.171.888 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.173.855 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 129 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.185.715 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 29, total running count: 29 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.186.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.189.413 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 130 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.200.805 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 30, total running count: 30 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.201.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.203.839 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 131 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.215.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 31, total running count: 31 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.216.613 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.218.599 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 132 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.230.582 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 32, total running count: 32 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.231.473 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.234.087 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 133 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.245.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 33, total running count: 33 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.246.629 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.248.583 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 134 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.260.567 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 34, total running count: 34 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.261.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.264.123 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 135 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.275.495 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 35, total running count: 35 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:28.276.484 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.278.276 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 136 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.290.666 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 36, total running count: 36 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.291.396 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.293.769 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 137 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.305.289 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 37, total running count: 37 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.306.395 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.308.115 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 138 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.320.346 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 38, total running count: 38 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.321.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.323.635 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 139 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.335.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 39, total running count: 39 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.336.177 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.338.208 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 140 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:28.350.183 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 40, total running count: 40 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:28.351.218 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.352.852 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 141 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.365.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 41, total running count: 41 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.366.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.368.708 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 142 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.380.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 42, total running count: 42 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.381.304 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.383.092 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 143 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.395.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 43, total running count: 43 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.396.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.398.678 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 144 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.410.439 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 44, total running count: 44 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.411.313 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.413.647 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 145 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.425.140 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 45, total running count: 45 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.426.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.428.131 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 146 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.440.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 46, total running count: 46 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.441.236 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.443.591 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 147 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.455.065 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 47, total running count: 47 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.456.246 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.458.979 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 148 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.470.173 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 48, total running count: 48 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.471.335 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.473.241 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 149 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.485.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 49, total running count: 49 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.486.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.488.709 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 150 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.500.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 50, total running count: 50 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.501.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.503.061 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 151 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.515.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 51, total running count: 51 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.516.441 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.518.864 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 152 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.530.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 52, total running count: 52 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.531.559 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.533.630 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 153 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.545.389 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 53, total running count: 53 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.546.746 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.549.326 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 154 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.560.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 54, total running count: 54 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.561.742 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.563.901 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 155 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.575.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 55, total running count: 55 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.576.691 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.578.498 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 156 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.590.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 56, total running count: 56 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.591.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.594.091 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 157 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:28.605.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 57, total running count: 57 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.606.736 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.608.539 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 158 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:28.620.514 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 58, total running count: 58 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.621.757 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.624.027 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 159 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.635.633 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 59, total running count: 59 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.636.748 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.638.384 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 160 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.650.716 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 60, total running count: 60 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.651.633 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.653.924 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 161 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.665.576 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 61, total running count: 61 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.666.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.668.626 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 162 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.680.398 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 62, total running count: 62 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.681.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.683.286 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 163 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.695.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 63, total running count: 63 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.696.527 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.698.689 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 164 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.710.476 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 64, total running count: 64 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.711.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.713.107 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 165 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.725.324 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 65, total running count: 65 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.726.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.728.563 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 166 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.740.521 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 66, total running count: 66 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.741.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.743.173 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 167 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.755.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 67, total running count: 67 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.756.403 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.759.043 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 168 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.770.292 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 68, total running count: 68 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.771.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.773.525 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 169 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.785.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 69, total running count: 69 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.786.231 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.788.003 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 170 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.800.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 70, total running count: 70 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.801.073 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.803.463 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 171 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.815.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 71, total running count: 71 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.816.079 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.818.003 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 172 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.830.013 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 72, total running count: 72 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.830.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.832.620 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 173 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.844.733 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 73, total running count: 73 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.845.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.848.239 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 174 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.859.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 74, total running count: 74 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.860.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.862.640 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 175 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.874.805 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 75, total running count: 75 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.875.756 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.877.341 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 176 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.889.813 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 76, total running count: 76 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.890.905 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.893.161 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 177 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.904.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 77, total running count: 77 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.905.770 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.907.545 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 178 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.919.514 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 78, total running count: 78 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:28.920.725 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.922.965 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 179 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.934.734 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 79, total running count: 79 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.935.704 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.937.419 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 180 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.949.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 80, total running count: 80 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.950.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.953.190 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 181 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.964.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 81, total running count: 81 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.965.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.967.817 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 182 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.979.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 82, total running count: 82 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:28.980.615 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.982.390 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 183 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:28.994.702 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 83, total running count: 83 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:28.995.680 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:28.998.031 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 184 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.009.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 84, total running count: 84 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.010.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.012.420 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 185 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.024.481 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 85, total running count: 85 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.025.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.027.932 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 186 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.039.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 86, total running count: 86 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.040.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.042.794 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 187 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.054.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 87, total running count: 87 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.055.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.057.061 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 188 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.069.191 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 88, total running count: 88 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.070.285 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.072.590 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 189 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.084.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 89, total running count: 89 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.085.375 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.086.969 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 190 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.099.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 90, total running count: 90 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:29.100.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.102.795 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 191 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.114.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 91, total running count: 91 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.115.286 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.117.370 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 192 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.129.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 92, total running count: 92 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.130.286 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.131.863 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 193 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.144.147 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 93, total running count: 93 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.145.349 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.147.553 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 194 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.159.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 94, total running count: 94 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.160.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.162.136 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 195 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.174.186 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 95, total running count: 95 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:29.175.270 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.176.949 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 196 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.189.150 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 96, total running count: 96 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:29.190.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.192.412 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 197 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.204.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 97, total running count: 97 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.205.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.206.745 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 198 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.219.269 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 98, total running count: 98 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.219.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.222.298 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 199 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.233.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 99, total running count: 99 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.234.926 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.236.570 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 200 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.248.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 100, total running count: 100 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.249.868 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.252.037 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 201 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.263.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 101, total running count: 101 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.264.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.266.322 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 202 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.278.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 102, total running count: 102 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:29.279.828 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.281.821 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 203 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.293.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 103, total running count: 103 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:29.294.688 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.297.400 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 204 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.308.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 104, total running count: 104 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.309.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.311.876 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 205 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.323.738 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 105, total running count: 105 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:29.324.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.326.641 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 206 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:29.338.551 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 106, total running count: 106 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.339.751 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.342.271 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 207 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.353.882 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 107, total running count: 107 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.354.586 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.356.647 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 208 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.368.385 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 108, total running count: 108 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:29.369.473 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.372.243 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 209 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.383.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 109, total running count: 109 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.384.459 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.386.578 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 210 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.398.145 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 110, total running count: 110 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.399.353 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.400.973 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 211 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:29.413.271 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 111, total running count: 111 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.414.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.416.596 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 212 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:29.428.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 112, total running count: 112 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.429.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.431.149 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 213 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.443.447 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 113, total running count: 113 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:29.444.404 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.446.976 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 214 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:29.458.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 114, total running count: 114 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:29.459.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.461.429 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 215 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.473.319 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 115, total running count: 115 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:29.474.420 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.476.986 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 216 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.488.398 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 116, total running count: 116 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.489.285 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.491.394 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 217 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.503.372 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 117, total running count: 117 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.504.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.506.007 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 218 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.518.374 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 118, total running count: 118 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:29.519.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.521.794 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 219 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.533.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 119, total running count: 119 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.534.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.536.387 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 220 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.548.249 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 120, total running count: 120 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.549.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.550.696 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 221 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.563.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 121, total running count: 121 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.564.102 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.566.397 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 222 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:29.578.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 122, total running count: 122 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.578.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.581.023 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 223 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.592.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 123, total running count: 123 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.593.936 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.595.763 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 224 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.607.777 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 124, total running count: 124 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.608.848 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.611.230 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 225 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.622.871 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 125, total running count: 125 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:29.623.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.625.613 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 226 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.637.909 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 126, total running count: 126 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:29.638.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.641.302 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 227 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.652.641 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 127, total running count: 127 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:29.653.888 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.655.659 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 228 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.667.798 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 128, total running count: 128 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.668.856 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.671.445 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 229 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.683.013 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 129, total running count: 129 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.684.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.686.167 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 230 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.698.042 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 130, total running count: 130 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.699.148 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.701.894 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 231 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.713.164 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 131, total running count: 131 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:29.714.208 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.716.246 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 232 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:29.728.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 132, total running count: 132 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.729.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.730.874 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 233 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.743.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 133, total running count: 133 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.744.187 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.746.316 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 234 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.758.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 134, total running count: 134 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.759.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.761.022 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 235 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.773.138 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 135, total running count: 135 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.774.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.776.784 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 236 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.788.042 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 136, total running count: 136 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.789.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.791.331 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 237 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:29.803.319 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 137, total running count: 137 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.804.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.806.002 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 238 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.818.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 138, total running count: 138 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.819.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.821.528 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 239 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.832.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 139, total running count: 139 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.834.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.835.946 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 240 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:29.848.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 140, total running count: 140 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.849.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.851.897 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 241 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.863.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 141, total running count: 141 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.864.105 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.866.709 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 242 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.878.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 142, total running count: 142 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:29.884.243 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.885.989 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 243 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.898.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 143, total running count: 143 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.899.246 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.901.985 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 244 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.913.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 144, total running count: 144 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:29.914.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.916.602 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 245 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:29.928.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 145, total running count: 145 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:29.929.162 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.930.914 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 246 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.943.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 146, total running count: 146 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.944.030 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.946.308 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 247 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:29.957.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 147, total running count: 147 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.959.013 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.960.628 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 248 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.972.942 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 148, total running count: 148 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:29.973.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.976.019 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 249 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:29.987.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 149, total running count: 149 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:29.988.922 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:29.991.456 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 250 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.003.013 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 150, total running count: 150 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.004.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.005.759 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 251 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.017.970 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 151, total running count: 151 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.018.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.021.382 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 252 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.033.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 152, total running count: 152 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.033.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.035.923 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 253 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.047.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 153, total running count: 153 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:30.049.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.051.814 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 254 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.063.089 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 154, total running count: 154 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.064.042 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.066.238 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 255 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.078.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 155, total running count: 155 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.079.052 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.081.778 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 256 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.093.054 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 156, total running count: 156 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.094.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.096.088 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 257 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.107.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 157, total running count: 157 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.108.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.111.421 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 258 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.122.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 158, total running count: 158 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.124.108 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.125.950 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 259 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.138.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 159, total running count: 159 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.139.048 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.141.668 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 260 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.153.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 160, total running count: 160 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.153.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.156.388 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 261 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.167.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 161, total running count: 161 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.168.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.170.912 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 262 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.182.915 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 162, total running count: 162 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:30.184.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.186.546 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 263 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.198.080 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 163, total running count: 163 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:30.199.135 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.201.025 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 264 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.213.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 164, total running count: 164 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.214.092 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.215.662 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 265 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.228.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 165, total running count: 165 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.228.964 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.231.541 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 266 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.243.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 166, total running count: 166 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.244.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.246.166 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 267 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.258.195 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 167, total running count: 167 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.259.204 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.261.970 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 268 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:30.273.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 168, total running count: 168 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.274.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.276.618 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 269 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:30.288.073 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 169, total running count: 169 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.289.199 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.290.938 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 270 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.303.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 170, total running count: 170 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.304.141 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.306.508 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 271 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:30.318.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 171, total running count: 171 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.319.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.320.997 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 272 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.333.348 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 172, total running count: 172 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.334.313 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.336.887 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 273 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.348.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 173, total running count: 173 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.349.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.351.479 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 274 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.363.208 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 174, total running count: 174 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.364.225 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.365.811 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 275 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.378.270 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 175, total running count: 175 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.379.092 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.381.339 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 276 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:30.393.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 176, total running count: 176 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.394.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.395.713 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 277 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.408.009 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 177, total running count: 177 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.409.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.411.220 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 278 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.423.142 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 178, total running count: 178 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.424.205 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.426.781 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 279 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.438.394 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 179, total running count: 179 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.439.184 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.441.272 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 280 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:30.453.120 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 180, total running count: 180 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.454.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.455.955 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 281 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.468.147 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 181, total running count: 181 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.469.086 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.471.442 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 282 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.483.037 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 182, total running count: 182 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.484.048 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.485.721 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 283 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.497.909 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 183, total running count: 183 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.499.027 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.501.136 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 284 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.512.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 184, total running count: 184 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.514.071 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.516.529 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 285 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:30.528.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 185, total running count: 185 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.529.077 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.530.846 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 286 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.543.149 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 186, total running count: 186 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.543.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.546.215 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 287 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.557.658 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 187, total running count: 187 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.558.701 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.560.557 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 288 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.572.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 188, total running count: 188 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.573.654 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.576.069 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 289 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.587.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 189, total running count: 189 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.588.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.590.283 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 290 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.602.391 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 190, total running count: 190 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.603.460 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.605.619 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 291 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.617.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 191, total running count: 191 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.618.347 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.620.006 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 292 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:30.632.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 192, total running count: 192 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.633.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.635.411 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 293 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.647.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 193, total running count: 193 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.648.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.649.788 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 294 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:30.661.968 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 194, total running count: 194 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.663.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.665.508 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 295 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.677.181 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 195, total running count: 195 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.678.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.680.144 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 296 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.692.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 196, total running count: 196 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.693.066 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.695.751 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 297 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:30.707.054 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 197, total running count: 197 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.707.938 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.710.098 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 298 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.721.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 198, total running count: 198 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:30.722.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.724.580 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 299 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.737.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 199, total running count: 199 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.737.899 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.740.041 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 300 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.751.738 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 200, total running count: 200 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.752.820 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.755.492 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 301 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.766.790 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 201, total running count: 201 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.767.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.769.831 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 302 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.781.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 202, total running count: 202 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.782.629 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.784.263 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 303 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.796.489 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 203, total running count: 203 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:30.797.471 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.800.084 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 304 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.811.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 204, total running count: 204 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.812.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.814.592 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 305 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.826.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 205, total running count: 205 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.827.362 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.829.125 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 306 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:30.841.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 206, total running count: 206 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:30.842.392 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.844.502 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 307 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:30.856.008 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 207, total running count: 207 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.857.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.858.773 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 308 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:30.871.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 208, total running count: 208 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:30.871.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.874.368 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 309 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.885.840 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 209, total running count: 209 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.886.930 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.888.788 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 310 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.900.632 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 210, total running count: 210 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.902.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.904.243 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 311 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.915.696 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 211, total running count: 211 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.916.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.918.584 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 312 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.930.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 212, total running count: 212 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.931.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.934.452 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 313 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:30.945.638 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 213, total running count: 213 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:30.946.681 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.948.671 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 314 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.960.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 214, total running count: 214 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:30.961.725 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.963.992 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 315 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.975.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 215, total running count: 215 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.976.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.978.199 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 316 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:30.990.618 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 216, total running count: 216 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:30.991.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:30.993.718 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 317 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:31.005.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 217, total running count: 217 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.006.474 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.008.050 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 318 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.020.625 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 218, total running count: 218 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.021.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.023.580 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 319 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.035.547 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 219, total running count: 219 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.036.558 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.039.076 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 320 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.050.622 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 220, total running count: 220 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.051.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.053.341 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 321 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.065.109 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 221, total running count: 221 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:31.066.375 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.067.964 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 322 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.080.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 222, total running count: 222 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.081.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.083.807 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 323 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.094.885 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 223, total running count: 223 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.096.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.098.251 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 324 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.109.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 224, total running count: 224 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.111.078 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.112.898 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 325 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.124.783 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 225, total running count: 225 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.126.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.127.636 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 326 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.139.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 226, total running count: 226 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.141.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.143.338 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 327 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.156.465 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 227, total running count: 227 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.156.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.158.878 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 328 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.170.928 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 228, total running count: 228 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.171.267 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.173.078 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 329 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.185.460 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 229, total running count: 229 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.186.035 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.188.555 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 330 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:31.200.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 230, total running count: 230 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:31.200.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.203.157 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 331 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:31.215.219 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 231, total running count: 231 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.216.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.217.599 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 332 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.230.051 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 232, total running count: 232 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.230.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.233.247 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 333 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.244.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 233, total running count: 233 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.245.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.247.581 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 334 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:31.259.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 234, total running count: 234 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.260.948 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.263.026 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 335 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.274.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 235, total running count: 235 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.275.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.278.639 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 336 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.289.877 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 236, total running count: 236 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.290.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.293.384 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 337 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.304.831 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 237, total running count: 237 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.306.002 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.308.043 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 338 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.319.827 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 238, total running count: 238 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.320.946 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.323.482 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 339 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.334.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 239, total running count: 239 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.336.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.337.779 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 340 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.349.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 240, total running count: 240 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.351.231 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.353.729 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 341 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:31.365.120 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 241, total running count: 241 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:31.366.338 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.368.478 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 342 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:31.380.318 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 242, total running count: 242 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:31.381.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.383.907 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 343 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.395.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 243, total running count: 243 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.396.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.398.383 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 344 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.410.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 244, total running count: 244 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.411.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.413.821 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 345 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:31.425.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 245, total running count: 245 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.426.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.428.209 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 346 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.440.300 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 246, total running count: 246 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.441.338 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.443.951 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 347 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:31.455.077 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 247, total running count: 247 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.456.251 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.458.695 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 348 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.470.264 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 248, total running count: 248 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.471.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.473.049 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 349 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.484.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 249, total running count: 249 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.486.163 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.488.755 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 350 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.500.168 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 250, total running count: 250 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.501.158 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.503.478 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 351 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:31.515.011 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 251, total running count: 251 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.516.203 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.517.951 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 352 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.530.131 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 252, total running count: 252 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.531.281 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.533.790 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 353 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.545.259 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 253, total running count: 253 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.546.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.548.663 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 354 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.560.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 254, total running count: 254 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.561.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.562.958 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 355 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:31.575.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 255, total running count: 255 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:31.576.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.578.579 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 356 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.590.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 256, total running count: 256 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.591.372 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.593.014 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 357 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.605.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 257, total running count: 257 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.606.939 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.609.012 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 358 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:31.620.707 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 258, total running count: 258 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.622.046 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.624.734 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 359 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.636.035 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 259, total running count: 259 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:31.637.204 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.638.886 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 360 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.651.009 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 260, total running count: 260 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.652.290 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.654.524 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 361 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.666.124 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 261, total running count: 261 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.667.336 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.668.897 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 362 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.681.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 262, total running count: 262 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.682.444 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.684.631 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 363 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.696.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 263, total running count: 263 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:31.697.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.699.166 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 364 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:31.711.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 264, total running count: 264 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.712.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.714.902 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 365 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:31.726.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 265, total running count: 265 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.727.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.729.406 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 366 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.741.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 266, total running count: 266 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.742.544 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.745.075 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 367 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.756.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 267, total running count: 267 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.757.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.759.575 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 368 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.771.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 268, total running count: 268 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:31.772.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.774.130 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 369 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.786.541 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 269, total running count: 269 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:31.787.442 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.789.776 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 370 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.801.381 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 270, total running count: 270 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.802.556 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.804.373 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 371 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.816.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 271, total running count: 271 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.817.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.819.084 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 372 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.831.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 272, total running count: 272 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.832.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.834.666 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 373 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.846.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 273, total running count: 273 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:31.847.416 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.849.025 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 374 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.861.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 274, total running count: 274 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.862.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.864.612 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 375 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.876.392 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 275, total running count: 275 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.877.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.878.985 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 376 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:31.891.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 276, total running count: 276 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.892.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.894.999 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 377 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.906.099 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 277, total running count: 277 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.907.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.909.494 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 378 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.920.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 278, total running count: 278 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.922.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.924.981 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 379 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.936.398 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 279, total running count: 279 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.937.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.939.388 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 380 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.951.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 280, total running count: 280 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.952.705 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.954.804 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 381 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.966.678 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 281, total running count: 281 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:31.967.700 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.970.278 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 382 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.981.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 282, total running count: 282 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:31.982.563 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:31.984.627 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 383 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:31.996.602 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 283, total running count: 283 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:31.997.584 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.000.222 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 384 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.011.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 284, total running count: 284 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.012.533 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.014.861 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 385 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.026.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 285, total running count: 285 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.027.394 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.029.378 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 386 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.041.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 286, total running count: 286 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.042.403 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.044.115 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 387 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.056.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 287, total running count: 287 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.057.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.059.893 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 388 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.071.286 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 288, total running count: 288 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.072.207 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.074.575 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 389 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.086.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 289, total running count: 289 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.087.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.089.295 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 390 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.100.945 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 290, total running count: 290 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.102.112 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.103.879 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 391 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.116.000 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 291, total running count: 291 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.116.946 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.118.632 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 392 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.130.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 292, total running count: 292 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.131.856 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.134.264 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 393 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.145.653 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 293, total running count: 293 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.146.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.148.657 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 394 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.160.569 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 294, total running count: 294 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.161.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.164.165 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 395 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.175.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 295, total running count: 295 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.176.619 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.178.832 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 396 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.190.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 296, total running count: 296 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.191.468 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.193.517 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 397 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.205.344 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 297, total running count: 297 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.206.510 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.208.993 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 398 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.220.486 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 298, total running count: 298 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.221.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.223.247 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 399 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.235.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 299, total running count: 299 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.236.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.238.728 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 400 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.250.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 300, total running count: 300 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.251.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.253.079 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 401 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.265.301 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 301, total running count: 301 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.266.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.268.542 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 402 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.280.171 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 302, total running count: 302 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.281.545 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.283.994 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 403 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.295.336 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 303, total running count: 303 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.296.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.298.324 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 404 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.310.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 304, total running count: 304 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.311.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.313.854 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 405 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.325.623 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 305, total running count: 305 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.326.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.328.120 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 406 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.340.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 306, total running count: 306 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.341.329 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.343.854 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 407 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.355.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 307, total running count: 307 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.356.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.358.436 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 408 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:32.370.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 308, total running count: 308 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.371.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.372.869 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 409 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.385.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 309, total running count: 309 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.386.331 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.388.620 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 410 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.400.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 310, total running count: 310 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.401.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.403.106 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 411 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.415.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 311, total running count: 311 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.416.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.418.921 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 412 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.430.286 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 312, total running count: 312 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.431.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.433.345 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 413 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.445.091 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 313, total running count: 313 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.446.290 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.448.006 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 414 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:32.460.028 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 314, total running count: 314 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:32.461.212 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.463.874 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 415 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.475.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 315, total running count: 315 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.476.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.478.254 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 416 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.490.226 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 316, total running count: 316 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.491.270 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.493.774 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 417 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.505.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 317, total running count: 317 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:32.506.264 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.508.304 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 418 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.520.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 318, total running count: 318 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.521.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.522.816 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 419 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.534.945 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 319, total running count: 319 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.536.086 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.538.649 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 420 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.550.048 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 320, total running count: 320 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.550.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.553.094 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 421 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.564.796 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 321, total running count: 321 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:32.566.030 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.568.626 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 422 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.579.594 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 322, total running count: 322 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.581.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.583.108 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 423 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.595.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 323, total running count: 323 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.596.035 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.597.650 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 424 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.609.746 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 324, total running count: 324 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.610.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.613.408 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 425 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.624.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 325, total running count: 325 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.625.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.627.769 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 426 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:32.640.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 326, total running count: 326 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.641.136 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.643.871 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 427 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.655.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 327, total running count: 327 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:32.656.437 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.658.689 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 428 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.670.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 328, total running count: 328 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:32.671.888 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.674.509 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 429 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.686.697 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 329, total running count: 329 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:32.687.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.689.310 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 430 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.702.056 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 330, total running count: 330 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.702.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.704.172 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 431 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.717.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 331, total running count: 331 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:32.717.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.720.155 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 432 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.732.378 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 332, total running count: 332 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.732.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.735.009 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 433 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.747.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 333, total running count: 333 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.747.851 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.749.850 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 434 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.762.270 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 334, total running count: 334 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.763.052 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.764.752 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 435 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.777.580 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 335, total running count: 335 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.778.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.780.918 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 436 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.792.762 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 336, total running count: 336 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.793.158 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.795.562 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 437 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.807.789 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 337, total running count: 337 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.808.174 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.810.180 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 438 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:32.822.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 338, total running count: 338 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:32.823.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.826.398 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 439 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.838.319 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 339, total running count: 339 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:32.838.840 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.841.292 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 440 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.853.476 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 340, total running count: 340 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.853.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.856.296 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 441 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:32.868.843 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 341, total running count: 341 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:32.869.236 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.871.228 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 442 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.883.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 342, total running count: 342 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.884.295 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.886.057 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 443 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:32.898.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 343, total running count: 343 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.899.303 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.901.905 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 444 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:32.914.001 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 344, total running count: 344 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:32.914.385 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.916.655 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 445 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.928.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 345, total running count: 345 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.929.324 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.931.373 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 446 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:32.943.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 346, total running count: 346 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.944.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.946.419 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 447 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:32.959.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 347, total running count: 347 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.959.584 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.962.372 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 448 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:32.974.148 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 348, total running count: 348 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:32.974.550 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.977.073 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 449 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:32.989.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 349, total running count: 349 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:32.989.719 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:32.991.579 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 450 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.004.378 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 350, total running count: 350 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.004.785 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.007.411 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 451 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.019.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 351, total running count: 351 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.019.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.021.934 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 452 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.034.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 352, total running count: 352 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.035.147 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.037.820 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 453 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:33.049.880 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 353, total running count: 353 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.050.249 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.052.637 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 454 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.064.845 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 354, total running count: 354 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.065.203 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.067.250 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 455 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.079.775 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 355, total running count: 355 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.080.130 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.082.830 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 456 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.094.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 356, total running count: 356 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.095.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.097.239 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 457 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.109.598 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 357, total running count: 357 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.110.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.112.781 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 458 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.124.692 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 358, total running count: 358 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.125.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.127.405 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 459 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:33.139.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 359, total running count: 359 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.140.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.141.762 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 460 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.154.721 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 360, total running count: 360 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.155.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.157.544 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 461 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.169.417 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 361, total running count: 361 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:33.170.187 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.172.046 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 462 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.184.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 362, total running count: 362 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.185.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.187.727 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 463 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.199.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 363, total running count: 363 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.200.285 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.202.400 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 464 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.214.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 364, total running count: 364 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.215.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.216.952 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 465 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.229.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 365, total running count: 365 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.230.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.232.708 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 466 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.244.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 366, total running count: 366 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.245.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.247.409 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 467 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.260.018 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 367, total running count: 367 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.260.380 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.262.021 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 468 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:33.274.865 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 368, total running count: 368 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.275.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.277.569 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 469 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:33.290.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 369, total running count: 369 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:33.290.374 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.292.064 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 470 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:33.305.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 370, total running count: 370 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.305.374 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.307.510 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 471 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.319.928 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 371, total running count: 371 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.320.344 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.321.951 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 472 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.334.942 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 372, total running count: 372 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.335.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.337.532 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 473 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.350.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 373, total running count: 373 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.350.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.353.114 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 474 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.364.946 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 374, total running count: 374 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.365.413 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.367.502 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 475 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.379.850 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 375, total running count: 375 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.380.290 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.381.837 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 476 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.394.584 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 376, total running count: 376 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:33.395.275 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.397.502 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 477 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.409.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 377, total running count: 377 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.410.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.413.072 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 478 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.424.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 378, total running count: 378 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.425.344 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.427.634 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 479 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.439.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 379, total running count: 379 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.440.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.442.335 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 480 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.454.777 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 380, total running count: 380 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.455.192 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.457.090 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 481 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.469.576 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 381, total running count: 381 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.470.186 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.472.567 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 482 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:33.484.619 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 382, total running count: 382 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.485.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.487.015 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 483 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.499.928 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 383, total running count: 383 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.500.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.502.660 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 484 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.514.948 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 384, total running count: 384 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.515.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.517.122 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 485 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.529.944 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 385, total running count: 385 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.530.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.532.723 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 486 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.544.933 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 386, total running count: 386 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:33.545.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.547.381 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 487 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.560.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 387, total running count: 387 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.560.402 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.562.135 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 488 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.574.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 388, total running count: 388 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.575.362 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.577.883 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 489 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.590.128 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 389, total running count: 389 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.590.545 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.592.179 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 490 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.605.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 390, total running count: 390 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.605.552 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.607.818 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 491 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.620.183 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 391, total running count: 391 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.620.550 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.622.364 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 492 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:33.635.331 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 392, total running count: 392 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.635.687 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.637.865 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 493 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.650.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 393, total running count: 393 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.650.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.652.516 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 494 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.665.111 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 394, total running count: 394 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.665.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.667.261 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 495 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.680.171 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 395, total running count: 395 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.680.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.682.926 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 496 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.695.116 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 396, total running count: 396 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.695.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.697.333 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 497 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.710.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 397, total running count: 397 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.710.551 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.712.795 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 498 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.725.037 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 398, total running count: 398 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.725.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.727.068 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 499 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.739.962 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 399, total running count: 399 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.740.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.742.807 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 500 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.754.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 400, total running count: 400 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.755.207 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.757.718 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 501 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.769.766 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 401, total running count: 401 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.770.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.772.361 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 502 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.784.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 402, total running count: 402 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.784.946 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.786.860 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 503 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.799.541 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 403, total running count: 403 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:33.799.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.802.412 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 504 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:33.814.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 404, total running count: 404 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.814.702 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.816.791 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 505 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.829.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 405, total running count: 405 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.829.756 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.831.466 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 506 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:33.844.466 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 406, total running count: 406 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.844.841 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.847.451 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 507 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.859.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 407, total running count: 407 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.859.707 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.861.911 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 508 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.874.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 408, total running count: 408 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.874.532 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.876.635 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 509 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.888.975 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 409, total running count: 409 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.889.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.890.998 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 510 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.904.061 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 410, total running count: 410 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.904.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.906.775 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 511 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.918.809 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 411, total running count: 411 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.919.158 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.921.365 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 512 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:33.933.793 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 412, total running count: 412 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.934.136 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.935.776 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 513 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:33.948.653 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 413, total running count: 413 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:33.949.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.951.367 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 514 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.963.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 414, total running count: 414 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.963.925 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.965.790 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 515 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.978.473 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 415, total running count: 415 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:33.978.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.981.437 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 516 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:33.993.511 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 416, total running count: 416 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:33.993.946 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:33.995.914 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 517 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.008.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 417, total running count: 417 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:34.008.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.010.822 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 518 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:34.023.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 418, total running count: 418 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.023.863 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.026.447 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 519 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.038.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 419, total running count: 419 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.038.665 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.040.750 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 520 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.053.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 420, total running count: 420 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.053.611 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.055.297 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 521 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.068.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 421, total running count: 421 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.068.679 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.070.956 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 522 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.083.051 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 422, total running count: 422 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.083.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.085.469 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 523 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.098.028 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 423, total running count: 423 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.098.392 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.100.114 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 524 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.112.975 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 424, total running count: 424 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:34.113.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.115.894 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 525 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.127.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 425, total running count: 425 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.128.319 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.130.221 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 526 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.142.909 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 426, total running count: 426 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.143.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.145.859 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 527 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.157.780 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 427, total running count: 427 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.158.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.160.101 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 528 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.172.594 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 428, total running count: 428 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.172.947 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.175.554 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 529 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:34.187.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 429, total running count: 429 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:34.187.831 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.190.252 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 530 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.202.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 430, total running count: 430 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.202.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.204.618 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 531 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.217.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 431, total running count: 431 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.217.719 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.219.448 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 532 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.232.232 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 432, total running count: 432 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.232.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.235.111 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 533 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.247.371 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 433, total running count: 433 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.247.734 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.249.809 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 534 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.262.286 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 434, total running count: 434 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.262.654 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.265.402 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 535 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.276.898 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 435, total running count: 435 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.277.250 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.278.815 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 536 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.291.948 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 436, total running count: 436 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.292.314 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.294.966 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 537 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.306.695 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 437, total running count: 437 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.307.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.309.390 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 538 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.321.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 438, total running count: 438 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.322.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.324.025 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 539 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.336.601 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 439, total running count: 439 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.336.968 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.338.597 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 540 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.351.424 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 440, total running count: 440 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.351.776 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.354.404 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 541 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:34.366.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 441, total running count: 441 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:34.366.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.368.968 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 542 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.381.163 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 442, total running count: 442 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.381.559 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.383.426 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 543 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.396.154 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 443, total running count: 443 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.396.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.399.222 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 544 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.411.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 444, total running count: 444 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.411.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.414.161 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 545 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:34.426.205 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 445, total running count: 445 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.426.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.428.626 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 546 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.441.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 446, total running count: 446 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.441.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.442.993 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 547 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:34.455.985 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 447, total running count: 447 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.456.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.458.749 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 548 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:34.470.853 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 448, total running count: 448 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.471.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.473.481 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 549 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:34.485.700 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 449, total running count: 449 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.486.100 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.488.227 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 550 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.500.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 450, total running count: 450 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.501.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.502.805 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 551 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.515.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 451, total running count: 451 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.516.013 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.518.588 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 552 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.530.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 452, total running count: 452 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.531.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.533.012 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 553 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.545.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 453, total running count: 453 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.545.858 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.548.533 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 554 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.560.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 454, total running count: 454 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.560.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.563.166 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 555 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.575.387 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 455, total running count: 455 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.575.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.577.500 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 556 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.590.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 456, total running count: 456 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.590.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.593.182 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 557 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.605.442 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 457, total running count: 457 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:34.605.860 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.607.761 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 558 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.620.219 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 458, total running count: 458 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.620.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.622.383 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 559 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.635.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 459, total running count: 459 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.635.831 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.638.295 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 560 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.650.195 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 460, total running count: 460 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.650.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.652.764 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 561 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.665.304 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 461, total running count: 461 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.665.722 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.668.219 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 562 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.680.174 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 462, total running count: 462 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.680.560 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.682.921 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 563 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.695.092 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 463, total running count: 463 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.695.483 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.697.599 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 564 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.710.111 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 464, total running count: 464 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.710.480 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.712.081 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 565 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.724.897 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 465, total running count: 465 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:34.725.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.727.779 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 566 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.739.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 466, total running count: 466 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.740.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.742.578 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 567 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.754.883 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 467, total running count: 467 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.755.251 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.757.069 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 568 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.769.999 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 468, total running count: 468 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:40:34.770.426 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_1 execution count: 1, execution time: 182128 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:40:34.770.661 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_1 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:40:34.770.814 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty epoch: 1 step: 468, loss is 2.3021485805511475 [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:40:34.772.046 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.h:76] ContinueSend] continue send at the beginning of the epoch [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:40:34.773.429 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:40:34.773.499 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:40:34.773.539 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_1 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.773.925 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.776.393 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 569 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.788.592 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 1, total running count: 469 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.788.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.790.796 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 570 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:34.803.651 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 2, total running count: 470 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.804.011 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.806.512 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 571 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.818.535 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 3, total running count: 471 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.818.880 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.820.915 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 572 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.833.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 4, total running count: 472 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.834.015 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.836.600 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 573 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.848.735 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 5, total running count: 473 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.849.099 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.851.209 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 574 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:34.863.708 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 6, total running count: 474 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.864.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.866.008 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 575 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.878.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 7, total running count: 475 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.879.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.881.579 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 576 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:34.893.717 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 8, total running count: 476 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.894.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.895.968 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 577 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.908.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 9, total running count: 477 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.909.052 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.911.469 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 578 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.923.539 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 10, total running count: 478 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.923.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.926.272 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 579 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.938.434 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 11, total running count: 479 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.938.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.940.731 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 580 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.953.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 12, total running count: 480 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.953.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.956.175 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 581 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.968.402 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 13, total running count: 481 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:34.968.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.971.073 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 582 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:34.983.433 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 14, total running count: 482 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:34.983.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:34.985.547 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 583 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.998.241 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 15, total running count: 483 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:34.998.592 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.001.067 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 584 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:35.013.031 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 16, total running count: 484 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.013.402 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.015.748 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 585 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.028.111 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 17, total running count: 485 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.028.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.030.596 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 586 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.043.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 18, total running count: 486 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.043.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.046.415 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 587 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.058.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 19, total running count: 487 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.058.645 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.060.663 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 588 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.073.080 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 20, total running count: 488 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.073.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.076.146 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 589 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.088.068 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 21, total running count: 489 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.088.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.090.583 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 590 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.102.829 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 22, total running count: 490 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.103.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.105.010 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 591 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.117.677 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 23, total running count: 491 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.118.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.119.820 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 592 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.132.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 24, total running count: 492 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.132.900 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.135.609 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 593 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.147.348 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 25, total running count: 493 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.147.698 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.150.168 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 594 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:35.164.149 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 26, total running count: 494 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.164.545 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.167.036 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 595 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.179.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 27, total running count: 495 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.179.868 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.181.525 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 596 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.194.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 28, total running count: 496 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.194.882 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.197.417 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 597 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.209.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 29, total running count: 497 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.209.813 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.211.999 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 598 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.224.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 30, total running count: 498 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.224.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.226.624 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 599 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.239.326 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 31, total running count: 499 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.239.688 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.242.345 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 600 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.254.100 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 32, total running count: 500 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.254.453 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.256.730 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 601 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:35.269.109 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 33, total running count: 501 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:35.269.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.272.229 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 602 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.284.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 34, total running count: 502 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.284.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.286.632 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 603 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.299.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 35, total running count: 503 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.299.543 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.302.254 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 604 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:35.314.251 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 36, total running count: 504 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.314.617 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.316.747 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 605 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.329.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 37, total running count: 505 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.329.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.331.399 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 606 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:35.343.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 38, total running count: 506 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.344.346 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.347.062 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 607 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.358.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 39, total running count: 507 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:35.359.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.361.766 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 608 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.373.762 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 40, total running count: 508 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.374.109 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.376.146 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 609 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.388.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 41, total running count: 509 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.388.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.390.667 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 610 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.403.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 42, total running count: 510 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.403.619 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.405.309 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 611 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.418.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 43, total running count: 511 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.418.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.420.823 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 612 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.433.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 44, total running count: 512 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:35.433.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.435.381 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 613 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.448.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 45, total running count: 513 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.448.644 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.451.340 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 614 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:35.463.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 46, total running count: 514 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.463.879 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.465.829 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 615 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.478.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 47, total running count: 515 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.478.885 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.481.404 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 616 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.493.489 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 48, total running count: 516 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.493.936 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.495.785 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 617 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:35.508.264 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 49, total running count: 517 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:35.508.752 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.511.309 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 618 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.524.647 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 50, total running count: 518 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:35.525.031 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.527.009 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 619 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:35.540.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 51, total running count: 519 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.540.883 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.542.703 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 620 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:35.555.770 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 52, total running count: 520 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.556.134 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.558.435 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 621 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.570.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 53, total running count: 521 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:35.571.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.574.033 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 622 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.586.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 54, total running count: 522 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.586.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.588.454 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 623 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.601.222 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 55, total running count: 523 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.601.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.603.808 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 624 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.616.402 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 56, total running count: 524 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.616.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.618.387 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 625 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.631.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 57, total running count: 525 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.631.775 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.634.183 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 626 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.646.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 58, total running count: 526 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.646.970 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.648.763 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 627 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.661.523 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 59, total running count: 527 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.661.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.664.207 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 628 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.676.412 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 60, total running count: 528 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.676.789 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.678.563 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 629 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.691.338 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 61, total running count: 529 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.691.704 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.694.435 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 630 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.706.205 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 62, total running count: 530 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.706.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.709.334 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 631 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.721.218 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 63, total running count: 531 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:35.721.596 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.723.773 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 632 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.736.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 64, total running count: 532 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.736.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.738.397 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 633 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.751.089 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 65, total running count: 533 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.751.444 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.753.254 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 634 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.765.930 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 66, total running count: 534 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.766.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.768.669 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 635 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.785.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 67, total running count: 535 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.785.642 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.787.873 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 636 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.801.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 68, total running count: 536 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.802.138 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.803.738 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 637 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.817.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 69, total running count: 537 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.817.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.819.403 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 638 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.832.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 70, total running count: 538 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.833.423 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.835.001 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 639 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.848.447 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 71, total running count: 539 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.849.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.851.874 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 640 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.864.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 72, total running count: 540 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.864.637 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.866.557 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 641 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:35.879.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 73, total running count: 541 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:35.879.947 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.882.297 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 642 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.894.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 74, total running count: 542 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.895.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.896.768 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 643 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:35.909.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 75, total running count: 543 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.910.260 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.912.481 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 644 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.924.916 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 76, total running count: 544 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.925.371 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.927.025 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 645 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.940.173 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 77, total running count: 545 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:35.940.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.942.920 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 646 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.955.105 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 78, total running count: 546 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:35.959.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.961.983 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 647 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.974.614 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 79, total running count: 547 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:35.975.032 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.976.622 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 648 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:35.989.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 80, total running count: 548 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:35.990.236 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:35.992.267 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 649 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.004.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 81, total running count: 549 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.005.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.007.967 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 650 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.020.016 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 82, total running count: 550 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:36.020.408 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.022.322 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 651 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.035.192 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 83, total running count: 551 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.035.608 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.037.848 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 652 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.050.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 84, total running count: 552 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.051.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.053.292 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 653 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.065.960 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 85, total running count: 553 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.066.370 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.068.736 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 654 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.080.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 86, total running count: 554 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.081.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.083.008 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 655 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.096.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 87, total running count: 555 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.097.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.099.792 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 656 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.111.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 88, total running count: 556 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.112.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.114.401 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 657 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.127.336 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 89, total running count: 557 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.127.762 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.129.893 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 658 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.142.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 90, total running count: 558 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.142.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.144.483 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 659 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.157.420 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 91, total running count: 559 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.157.890 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.160.043 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 660 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.172.371 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 92, total running count: 560 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.172.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.174.594 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 661 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:36.187.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 93, total running count: 561 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.188.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.190.416 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 662 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:36.203.068 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 94, total running count: 562 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.203.444 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.205.069 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 663 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.218.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 95, total running count: 563 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.218.532 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.220.700 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 664 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.233.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 96, total running count: 564 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.233.551 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.236.147 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 665 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.248.307 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 97, total running count: 565 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.248.700 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.250.776 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 666 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.263.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 98, total running count: 566 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.263.665 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.265.473 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 667 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.278.358 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 99, total running count: 567 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.278.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.281.586 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 668 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.293.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 100, total running count: 568 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.293.978 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.296.299 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 669 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.308.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 101, total running count: 569 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.308.949 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.310.974 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 670 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:36.323.408 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 102, total running count: 570 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.323.788 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.325.423 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 671 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.338.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 103, total running count: 571 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:36.338.903 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.341.007 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 672 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.353.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 104, total running count: 572 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.354.068 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.356.661 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 673 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.368.766 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 105, total running count: 573 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.369.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.371.363 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 674 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:36.383.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 106, total running count: 574 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:36.384.324 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.386.958 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 675 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.399.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 107, total running count: 575 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.399.440 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.401.243 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 676 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:36.414.065 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 108, total running count: 576 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.414.944 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.416.760 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 677 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.429.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 109, total running count: 577 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:36.430.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.433.112 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 678 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.445.514 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 110, total running count: 578 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.446.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.448.697 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 679 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.461.417 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 111, total running count: 579 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.462.589 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.465.260 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 680 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.477.356 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 112, total running count: 580 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.478.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.480.046 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 681 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.492.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 113, total running count: 581 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.493.680 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.495.808 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 682 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.508.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 114, total running count: 582 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.514.096 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.516.060 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 683 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.528.691 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 115, total running count: 583 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.531.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.532.822 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 684 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.545.933 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 116, total running count: 584 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.546.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.549.412 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 685 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.561.232 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 117, total running count: 585 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:36.562.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.565.270 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 686 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.577.173 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 118, total running count: 586 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.578.243 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.579.981 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 687 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.592.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 119, total running count: 587 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.593.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.595.456 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 688 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.608.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 120, total running count: 588 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.609.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.610.943 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 689 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.623.915 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 121, total running count: 589 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.625.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.627.518 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 690 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.639.887 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 122, total running count: 590 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.641.043 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.642.951 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 691 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:36.655.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 123, total running count: 591 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.656.897 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.659.541 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 692 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.671.725 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 124, total running count: 592 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.672.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.674.937 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 693 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:36.687.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 125, total running count: 593 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.688.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.690.436 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 694 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.702.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 126, total running count: 594 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.703.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.705.936 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 695 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:36.718.417 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 127, total running count: 595 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.719.535 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.721.357 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 696 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.734.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 128, total running count: 596 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.736.437 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.738.160 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 697 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.751.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 129, total running count: 597 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:36.751.943 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.753.892 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 698 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.766.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 130, total running count: 598 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.767.765 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.769.865 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 699 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.782.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 131, total running count: 599 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.783.434 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.785.427 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 700 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.797.997 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 132, total running count: 600 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.799.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.800.939 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 701 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.813.765 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 133, total running count: 601 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.814.960 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.817.606 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 702 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.829.629 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 134, total running count: 602 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.830.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.833.074 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 703 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.844.966 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 135, total running count: 603 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.848.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.850.860 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 704 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.863.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 136, total running count: 604 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.864.822 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.867.494 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 705 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.879.460 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 137, total running count: 605 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:36.881.018 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.883.108 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 706 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.895.650 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 138, total running count: 606 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.896.678 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.898.854 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 707 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.911.344 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 139, total running count: 607 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.912.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.914.766 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 708 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.926.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 140, total running count: 608 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.928.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.930.283 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 709 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.943.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 141, total running count: 609 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.949.092 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.951.521 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 710 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.963.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 142, total running count: 610 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.964.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.966.112 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 711 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.978.945 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 143, total running count: 611 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:36.980.225 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.981.878 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 712 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:36.994.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 144, total running count: 612 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:36.995.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:36.997.905 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 713 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.010.620 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 145, total running count: 613 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.011.707 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.013.488 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 714 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.026.455 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 146, total running count: 614 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.027.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.029.334 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 715 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.042.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 147, total running count: 615 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.042.985 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.044.962 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 716 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.057.811 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 148, total running count: 616 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.058.717 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.060.523 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 717 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.073.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 149, total running count: 617 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:37.074.559 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.076.225 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 718 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.089.147 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 150, total running count: 618 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.090.539 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.093.130 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 719 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:37.105.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 151, total running count: 619 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:37.106.508 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.108.911 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 720 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.121.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 152, total running count: 620 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.122.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.124.492 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 721 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.137.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 153, total running count: 621 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.138.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.140.030 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 722 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.152.670 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 154, total running count: 622 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:37.153.832 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.155.638 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 723 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.168.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 155, total running count: 623 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.169.680 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.171.289 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 724 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.184.539 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 156, total running count: 624 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.185.132 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.186.828 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 725 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:37.199.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 157, total running count: 625 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.200.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.202.368 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 726 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.215.510 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 158, total running count: 626 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.216.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.218.953 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 727 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.231.009 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 159, total running count: 627 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.232.107 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.234.420 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 728 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.246.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 160, total running count: 628 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.247.693 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.249.919 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 729 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.262.300 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 161, total running count: 629 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.263.399 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.265.656 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 730 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.278.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 162, total running count: 630 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.278.783 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.281.426 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 731 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.293.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 163, total running count: 631 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.294.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.296.969 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 732 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.309.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 164, total running count: 632 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.310.131 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.312.538 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 733 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:37.324.734 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 165, total running count: 633 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:37.325.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.328.063 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 734 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:37.340.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 166, total running count: 634 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.341.597 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.343.854 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 735 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.356.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 167, total running count: 635 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.357.271 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.359.871 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 736 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.371.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 168, total running count: 636 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.372.791 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.375.383 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 737 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.387.246 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 169, total running count: 637 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.388.423 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.391.119 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 738 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.403.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 170, total running count: 638 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.404.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.406.990 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 739 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.419.181 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 171, total running count: 639 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.420.163 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.422.613 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 740 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.434.651 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 172, total running count: 640 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:37.435.764 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.438.425 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 741 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.450.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 173, total running count: 641 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.451.636 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.454.330 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 742 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.466.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 174, total running count: 642 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.467.484 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.469.861 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 743 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.482.173 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 175, total running count: 643 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.483.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.485.375 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 744 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.497.951 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 176, total running count: 644 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.498.708 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.501.036 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 745 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.513.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 177, total running count: 645 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.514.413 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.516.941 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 746 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:37.529.039 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 178, total running count: 646 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.529.762 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.531.456 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 747 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.544.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 179, total running count: 647 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.545.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.548.061 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 748 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.559.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 180, total running count: 648 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.560.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.563.533 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 749 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.575.562 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 181, total running count: 649 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:37.576.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.578.924 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 750 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.590.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 182, total running count: 650 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.591.878 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.593.676 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 751 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.606.522 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 183, total running count: 651 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:37.607.727 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.609.402 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 752 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.622.412 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 184, total running count: 652 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:37.623.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.625.108 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 753 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.637.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 185, total running count: 653 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:37.639.045 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.641.767 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 754 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.653.400 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 186, total running count: 654 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.654.839 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.656.501 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 755 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.669.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 187, total running count: 655 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.670.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.673.431 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 756 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.685.425 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 188, total running count: 656 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.686.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.687.967 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 757 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:37.700.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 189, total running count: 657 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.701.998 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.703.960 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 758 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.716.569 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 190, total running count: 658 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.717.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.719.491 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 759 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.731.872 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 191, total running count: 659 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.732.970 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.734.962 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 760 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.747.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 192, total running count: 660 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.748.580 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.750.551 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 761 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.763.135 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 193, total running count: 661 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:37.764.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.766.553 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 762 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.778.663 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 194, total running count: 662 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.779.829 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.782.265 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 763 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.794.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 195, total running count: 663 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.795.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.797.929 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 764 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:37.810.073 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 196, total running count: 664 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:37.811.202 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.813.870 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 765 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.825.844 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 197, total running count: 665 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:37.826.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.829.647 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 766 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.841.721 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 198, total running count: 666 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.842.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.844.060 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 767 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.857.136 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 199, total running count: 667 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.858.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.860.878 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 768 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.872.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 200, total running count: 668 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.873.900 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.876.544 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 769 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:37.888.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 201, total running count: 669 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.889.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.892.269 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 770 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.904.346 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 202, total running count: 670 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.905.187 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.906.863 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 771 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.919.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 203, total running count: 671 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:37.921.200 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.923.529 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 772 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.935.752 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 204, total running count: 672 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.936.776 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.938.930 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 773 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.951.493 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 205, total running count: 673 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:37.952.317 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.954.612 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 774 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:37.966.717 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 206, total running count: 674 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:37.967.949 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.970.297 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 775 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:37.982.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 207, total running count: 675 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.983.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:37.986.198 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 776 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:37.998.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 208, total running count: 676 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:37.999.028 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.000.616 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 777 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.013.547 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 209, total running count: 677 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.014.794 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.017.333 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 778 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.029.292 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 210, total running count: 678 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.030.587 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.033.017 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 779 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.045.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 211, total running count: 679 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.046.241 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.048.775 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 780 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.060.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 212, total running count: 680 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:38.061.759 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.064.425 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 781 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:38.076.207 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 213, total running count: 681 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.077.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.078.892 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 782 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.091.837 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 214, total running count: 682 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.092.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.094.403 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 783 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.107.510 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 215, total running count: 683 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.108.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.111.085 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 784 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.123.399 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 216, total running count: 684 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.124.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.126.911 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 785 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.139.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 217, total running count: 685 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.139.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.141.744 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 786 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.154.551 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 218, total running count: 686 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.155.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.157.270 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 787 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.170.225 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 219, total running count: 687 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:38.171.089 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.172.667 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 788 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.185.535 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 220, total running count: 688 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.186.617 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.188.242 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 789 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.201.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 221, total running count: 689 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.202.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.203.984 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 790 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.216.798 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 222, total running count: 690 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.217.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.219.804 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 791 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.232.532 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 223, total running count: 691 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.233.560 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.235.256 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 792 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.248.210 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 224, total running count: 692 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.248.992 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.250.860 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 793 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.263.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 225, total running count: 693 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.264.441 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.266.345 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 794 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.279.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 226, total running count: 694 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.279.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.281.871 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 795 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.294.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 227, total running count: 695 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.295.599 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.297.558 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 796 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.309.937 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 228, total running count: 696 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.311.080 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.313.578 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 797 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.325.550 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 229, total running count: 697 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.326.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.328.062 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 798 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.341.138 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 230, total running count: 698 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.342.252 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.344.897 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 799 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.356.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 231, total running count: 699 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.357.829 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.359.490 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 800 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.372.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 232, total running count: 700 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.373.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.374.980 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 801 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.387.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 233, total running count: 701 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.388.809 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.390.379 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 802 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.403.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 234, total running count: 702 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.404.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.407.076 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 803 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.419.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 235, total running count: 703 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.419.887 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.422.614 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 804 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:38.434.563 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 236, total running count: 704 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.435.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.437.314 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 805 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.449.926 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 237, total running count: 705 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.450.732 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.453.269 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 806 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.465.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 238, total running count: 706 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.466.142 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.467.941 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 807 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.480.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 239, total running count: 707 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.481.808 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.483.868 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 808 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.496.285 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 240, total running count: 708 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.497.275 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.499.443 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 809 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.511.755 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 241, total running count: 709 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.512.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.515.491 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 810 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.527.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 242, total running count: 710 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:38.528.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.529.946 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 811 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.542.797 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 243, total running count: 711 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.543.938 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.545.575 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 812 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.558.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 244, total running count: 712 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.559.541 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.561.140 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 813 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.574.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 245, total running count: 713 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.575.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.577.801 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 814 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:38.589.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 246, total running count: 714 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.590.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.593.296 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 815 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:38.605.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 247, total running count: 715 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.606.594 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.608.830 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 816 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.621.045 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 248, total running count: 716 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:38.622.214 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.624.618 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 817 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.636.800 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 249, total running count: 717 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.637.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.640.460 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 818 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.652.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 250, total running count: 718 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.653.632 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.656.032 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 819 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.668.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 251, total running count: 719 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.669.158 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.671.581 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 820 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.683.683 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 252, total running count: 720 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.684.812 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.686.966 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 821 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.699.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 253, total running count: 721 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.700.325 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.702.535 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 822 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:38.714.871 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 254, total running count: 722 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.715.793 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.718.057 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 823 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.730.466 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 255, total running count: 723 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.731.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.733.663 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 824 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.745.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 256, total running count: 724 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.746.529 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.748.334 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 825 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:38.761.080 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 257, total running count: 725 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.762.018 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.763.996 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 826 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.776.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 258, total running count: 726 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.777.263 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.779.537 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 827 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.792.042 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 259, total running count: 727 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.792.949 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.795.094 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 828 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.807.543 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 260, total running count: 728 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.808.396 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.810.963 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 829 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.823.043 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 261, total running count: 729 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:38.823.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.825.630 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 830 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:38.838.374 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 262, total running count: 730 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.839.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.841.282 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 831 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.854.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 263, total running count: 731 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.855.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.857.083 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 832 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.869.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 264, total running count: 732 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.870.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.873.087 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 833 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:38.885.039 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 265, total running count: 733 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.885.937 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.888.497 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 834 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.900.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 266, total running count: 734 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:38.901.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.902.928 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 835 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.915.992 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 267, total running count: 735 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.916.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.919.751 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 836 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.931.620 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 268, total running count: 736 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:38.932.388 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.934.582 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 837 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:38.946.989 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 269, total running count: 737 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.948.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.950.257 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 838 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.962.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 270, total running count: 738 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:38.963.714 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.965.973 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 839 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.978.336 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 271, total running count: 739 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:38.979.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.981.857 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 840 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:38.993.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 272, total running count: 740 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:38.994.829 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:38.996.542 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 841 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.009.478 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 273, total running count: 741 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:39.010.510 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.013.275 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 842 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.025.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 274, total running count: 742 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.026.092 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.028.716 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 843 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.040.796 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 275, total running count: 743 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.041.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.044.445 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 844 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:39.055.970 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 276, total running count: 744 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:39.057.335 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.059.026 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 845 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.071.854 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 277, total running count: 745 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.072.789 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.074.894 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 846 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.087.326 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 278, total running count: 746 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.088.530 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.090.619 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 847 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.103.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 279, total running count: 747 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:39.104.056 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.106.571 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 848 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.118.752 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 280, total running count: 748 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.119.583 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.122.167 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 849 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.134.148 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 281, total running count: 749 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.135.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.137.721 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 850 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.149.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 282, total running count: 750 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.150.668 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.152.256 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 851 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.165.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 283, total running count: 751 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:39.166.118 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.167.898 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 852 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.180.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 284, total running count: 752 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.181.521 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.183.470 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 853 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.196.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 285, total running count: 753 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.196.945 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.199.293 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 854 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.211.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 286, total running count: 754 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:39.212.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.214.885 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 855 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.227.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 287, total running count: 755 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.227.967 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.230.535 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 856 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.242.686 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 288, total running count: 756 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:39.243.790 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.245.391 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 857 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.258.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 289, total running count: 757 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.259.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.262.291 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 858 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.274.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 290, total running count: 758 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.275.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.276.723 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 859 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.289.683 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 291, total running count: 759 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.290.690 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.292.433 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 860 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.305.329 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 292, total running count: 760 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.306.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.308.286 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 861 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.320.770 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 293, total running count: 761 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:39.321.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.324.071 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 862 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.336.496 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 294, total running count: 762 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.337.246 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.339.672 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 863 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.351.765 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 295, total running count: 763 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.352.779 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.355.083 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 864 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.367.231 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 296, total running count: 764 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.368.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.370.802 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 865 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.382.930 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 297, total running count: 765 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.383.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.385.630 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 866 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.398.517 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 298, total running count: 766 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.399.132 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.401.263 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 867 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.413.720 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 299, total running count: 767 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.414.848 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.416.657 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 868 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.429.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 300, total running count: 768 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.430.275 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.432.224 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 869 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.444.812 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 301, total running count: 769 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.446.008 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.447.923 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 870 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.460.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 302, total running count: 770 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:39.461.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.463.785 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 871 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.476.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 303, total running count: 771 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.476.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.479.521 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 872 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.491.612 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 304, total running count: 772 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.492.641 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.495.290 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 873 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:39.507.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 305, total running count: 773 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.508.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.509.922 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 874 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.522.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 306, total running count: 774 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.523.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.525.785 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 875 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.538.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 307, total running count: 775 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.539.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.541.349 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 876 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.553.916 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 308, total running count: 776 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.554.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.556.843 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 877 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.569.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 309, total running count: 777 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.570.343 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.572.476 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 878 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.585.011 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 310, total running count: 778 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.585.680 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.588.280 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 879 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:39.600.217 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 311, total running count: 779 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.601.344 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.603.876 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 880 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.615.989 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 312, total running count: 780 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.616.841 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.619.359 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 881 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.631.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 313, total running count: 781 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.632.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.634.864 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 882 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.646.951 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 314, total running count: 782 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.648.184 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.650.325 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 883 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.662.948 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 315, total running count: 783 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.663.695 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.665.783 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 884 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.678.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 316, total running count: 784 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.679.216 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.681.200 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 885 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.693.799 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 317, total running count: 785 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:39.694.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.696.727 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 886 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.709.173 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 318, total running count: 786 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:39.710.214 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.712.164 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 887 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.724.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 319, total running count: 787 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.725.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.727.922 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 888 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.740.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 320, total running count: 788 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:39.740.975 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.742.748 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 889 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.755.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 321, total running count: 789 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.756.478 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.758.243 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 890 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:39.771.073 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 322, total running count: 790 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.772.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.773.780 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 891 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.786.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 323, total running count: 791 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.787.371 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.789.328 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 892 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.801.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 324, total running count: 792 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.802.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.804.719 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 893 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.817.651 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 325, total running count: 793 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.818.484 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.820.550 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 894 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.833.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 326, total running count: 794 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.834.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.836.351 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 895 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.848.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 327, total running count: 795 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:39.850.079 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.852.052 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 896 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.864.758 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 328, total running count: 796 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.865.809 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.868.011 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 897 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.880.552 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 329, total running count: 797 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.881.437 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.883.811 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 898 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.896.156 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 330, total running count: 798 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.897.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.899.717 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 899 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.912.508 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 331, total running count: 799 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.913.301 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.915.483 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 900 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.928.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 332, total running count: 800 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.928.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.931.347 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 901 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.943.720 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 333, total running count: 801 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.944.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.947.267 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 902 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:39.959.348 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 334, total running count: 802 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:39.960.437 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.963.090 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 903 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.975.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 335, total running count: 803 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.976.205 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.977.948 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 904 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:39.990.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 336, total running count: 804 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:39.991.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:39.993.722 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 905 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:40.006.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 337, total running count: 805 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.007.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.009.726 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 906 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.022.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 338, total running count: 806 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.023.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.025.452 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 907 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:40.038.395 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 339, total running count: 807 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.039.584 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.041.313 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 908 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.054.030 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 340, total running count: 808 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.055.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.057.317 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 909 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.069.939 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 341, total running count: 809 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.070.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.073.281 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 910 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.085.809 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 342, total running count: 810 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.086.674 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.088.978 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 911 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.101.343 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 343, total running count: 811 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.102.413 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.104.941 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 912 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.117.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 344, total running count: 812 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.118.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.119.831 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 913 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.132.860 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 345, total running count: 813 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.134.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.136.702 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 914 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.149.134 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 346, total running count: 814 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.150.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.152.428 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 915 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.164.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 347, total running count: 815 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.166.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.168.006 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 916 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:40.180.722 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 348, total running count: 816 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.181.527 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.183.777 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 917 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.196.248 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 349, total running count: 817 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.197.048 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.199.496 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 918 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.211.808 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 350, total running count: 818 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.212.815 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.215.114 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 919 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:40.227.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 351, total running count: 819 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.228.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.230.850 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 920 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.242.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 352, total running count: 820 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:40.244.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.246.438 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 921 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:40.258.630 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 353, total running count: 821 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.259.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.262.191 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 922 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.274.350 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 354, total running count: 822 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.275.336 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.276.945 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 923 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.289.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 355, total running count: 823 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.290.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.292.689 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 924 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.305.265 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 356, total running count: 824 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.306.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.308.378 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 925 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.320.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 357, total running count: 825 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.321.938 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.323.916 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 926 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.336.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 358, total running count: 826 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.337.641 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.339.847 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 927 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.352.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 359, total running count: 827 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.353.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.355.353 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 928 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.367.609 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 360, total running count: 828 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.368.998 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.370.891 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 929 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.383.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 361, total running count: 829 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.384.441 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.386.556 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 930 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.399.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 362, total running count: 830 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.399.985 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.402.329 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 931 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.414.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 363, total running count: 831 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.415.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.418.041 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 932 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.429.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 364, total running count: 832 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.430.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.432.730 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 933 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.445.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 365, total running count: 833 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.446.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.448.527 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 934 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.460.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 366, total running count: 834 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.461.985 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.463.919 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 935 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:40.476.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 367, total running count: 835 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.477.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.479.548 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 936 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.491.690 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 368, total running count: 836 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.492.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.495.116 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 937 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.507.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 369, total running count: 837 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.508.391 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.510.797 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 938 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:40.522.965 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 370, total running count: 838 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.524.077 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.526.364 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 939 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.538.645 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 371, total running count: 839 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.539.587 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.541.847 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 940 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.553.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 372, total running count: 840 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.555.186 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.557.414 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 941 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.569.832 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 373, total running count: 841 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.570.903 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.573.333 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 942 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.585.468 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 374, total running count: 842 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:40.586.551 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.588.169 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 943 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.601.155 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 375, total running count: 843 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.602.232 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.604.086 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 944 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.616.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 376, total running count: 844 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.617.832 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.619.861 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 945 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.632.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 377, total running count: 845 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.633.225 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.635.745 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 946 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:40.647.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 378, total running count: 846 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.648.971 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.651.159 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 947 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.663.495 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 379, total running count: 847 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.664.860 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.666.752 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 948 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.679.433 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 380, total running count: 848 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.680.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.682.163 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 949 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.695.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 381, total running count: 849 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:40.695.964 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.698.419 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 950 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.710.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 382, total running count: 850 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.711.756 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.714.146 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 951 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.726.007 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 383, total running count: 851 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.727.473 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.729.626 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 952 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.741.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 384, total running count: 852 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.743.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.745.591 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 953 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:40.757.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 385, total running count: 853 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.758.675 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.761.496 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 954 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.773.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 386, total running count: 854 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.774.432 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.776.240 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 955 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.789.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 387, total running count: 855 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.789.915 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.791.973 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 956 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.804.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 388, total running count: 856 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.805.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.807.660 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 957 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.819.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 389, total running count: 857 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.820.968 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.823.554 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 958 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.835.633 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 390, total running count: 858 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.836.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.839.525 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 959 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.851.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 391, total running count: 859 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.852.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.854.091 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 960 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.867.097 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 392, total running count: 860 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.867.973 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.869.715 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 961 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.882.468 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 393, total running count: 861 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.883.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.885.253 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 962 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.898.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 394, total running count: 862 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.899.231 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.900.842 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 963 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.913.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 395, total running count: 863 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.914.649 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.916.584 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 964 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.929.153 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 396, total running count: 864 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.930.181 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.932.170 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 965 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.944.709 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 397, total running count: 865 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:40.945.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.947.766 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 966 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.960.184 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 398, total running count: 866 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:40.961.547 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.963.561 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 967 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:40.976.079 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 399, total running count: 867 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.977.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.979.181 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 968 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:40.991.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 400, total running count: 868 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:40.992.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:40.994.752 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 969 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.007.148 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 401, total running count: 869 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.008.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.010.664 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 970 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.022.880 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 402, total running count: 870 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.023.765 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.026.236 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 971 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.038.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 403, total running count: 871 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.039.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.041.841 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 972 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.053.937 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 404, total running count: 872 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.054.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.057.319 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 973 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.069.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 405, total running count: 873 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.070.602 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.072.700 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 974 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.085.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 406, total running count: 874 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:41.086.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.088.328 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 975 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.100.757 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 407, total running count: 875 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.101.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.103.791 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 976 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.116.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 408, total running count: 876 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.117.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.119.423 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 977 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.132.065 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 409, total running count: 877 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.133.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.134.938 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 978 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.147.686 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 410, total running count: 878 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.148.480 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.150.823 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 979 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.162.856 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 411, total running count: 879 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.163.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.166.680 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 980 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.178.443 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 412, total running count: 880 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.179.471 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.181.438 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 981 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.194.138 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 413, total running count: 881 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.194.899 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.197.425 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 982 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.209.719 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 414, total running count: 882 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.210.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.211.960 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 983 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.224.938 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 415, total running count: 883 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.225.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.227.694 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 984 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.240.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 416, total running count: 884 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.241.395 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.243.253 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 985 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.255.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 417, total running count: 885 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.257.112 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.258.855 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 986 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.271.638 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 418, total running count: 886 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.272.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.274.397 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 987 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.287.056 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 419, total running count: 887 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.288.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.290.178 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 988 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.302.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 420, total running count: 888 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.303.828 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.305.758 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 989 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.318.380 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 421, total running count: 889 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.319.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.321.578 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 990 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.333.860 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 422, total running count: 890 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.335.304 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.337.517 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 991 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.349.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 423, total running count: 891 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.350.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.353.299 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 992 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:41.365.295 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 424, total running count: 892 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.366.378 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.368.050 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 993 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.380.799 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 425, total running count: 893 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.381.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.383.969 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 994 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.396.486 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 426, total running count: 894 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.397.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.399.636 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 995 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.412.030 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 427, total running count: 895 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.413.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.415.334 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 996 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.427.675 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 428, total running count: 896 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:41.428.657 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.430.939 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 997 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.443.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 429, total running count: 897 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.444.048 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.445.774 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 998 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.458.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 430, total running count: 898 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.459.690 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.461.446 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 999 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.474.203 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 431, total running count: 899 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.475.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.477.370 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1000 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.489.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 432, total running count: 900 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:41.490.525 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.492.118 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1001 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.504.798 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 433, total running count: 901 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.506.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.508.155 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1002 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.520.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 434, total running count: 902 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.521.792 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.524.064 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1003 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:41.536.124 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 435, total running count: 903 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.537.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.539.728 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1004 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.551.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 436, total running count: 904 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.552.944 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.555.361 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1005 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.567.301 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 437, total running count: 905 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.568.586 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.571.159 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1006 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.583.285 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 438, total running count: 906 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.584.250 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.586.683 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1007 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.598.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 439, total running count: 907 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.599.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.602.121 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1008 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.614.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 440, total running count: 908 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.615.219 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.617.741 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1009 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.629.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 441, total running count: 909 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.630.660 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.633.340 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1010 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.645.162 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 442, total running count: 910 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:41.646.329 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.647.961 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1011 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.660.876 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 443, total running count: 911 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.661.975 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.663.904 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1012 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.676.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 444, total running count: 912 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:41.677.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.679.576 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1013 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.691.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 445, total running count: 913 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:41.693.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.695.453 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1014 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.707.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 446, total running count: 914 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.708.731 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.711.414 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1015 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.723.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 447, total running count: 915 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.724.358 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.726.195 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1016 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.738.839 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 448, total running count: 916 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.739.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.741.784 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1017 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.754.372 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 449, total running count: 917 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.755.484 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.757.675 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1018 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.769.633 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 450, total running count: 918 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.770.916 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.772.573 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1019 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.785.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 451, total running count: 919 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.786.439 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.788.277 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1020 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.801.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 452, total running count: 920 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:41.802.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.803.975 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1021 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.816.625 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 453, total running count: 921 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:41.817.736 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.819.778 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1022 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.832.280 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 454, total running count: 922 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.833.471 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.835.793 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1023 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:41.847.804 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 455, total running count: 923 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.849.347 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.851.512 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1024 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.863.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 456, total running count: 924 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:41.864.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.867.264 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1025 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.879.391 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 457, total running count: 925 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.880.457 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.882.152 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1026 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.895.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 458, total running count: 926 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:41.896.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.897.892 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1027 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.910.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 459, total running count: 927 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.911.559 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.913.809 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1028 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.926.127 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 460, total running count: 928 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.927.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.929.682 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1029 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.941.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 461, total running count: 929 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:41.942.829 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.945.450 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1030 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.957.263 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 462, total running count: 930 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:41.958.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.960.996 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1031 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.972.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 463, total running count: 931 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:41.974.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.975.787 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1032 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.988.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 464, total running count: 932 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:41.989.678 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:41.991.497 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1033 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.004.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 465, total running count: 933 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.005.239 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.007.312 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1034 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.019.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 466, total running count: 934 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.020.900 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.022.972 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1035 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:42.035.547 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 467, total running count: 935 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.036.375 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.038.586 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1036 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.050.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 468, total running count: 936 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:40:42.052.315 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_1 execution count: 2, execution time: 7278.64 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:40:42.052.492 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_1 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:40:42.052.577 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty epoch: 2 step: 468, loss is 2.3135838508605957 [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:40:42.053.558 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.h:76] ContinueSend] continue send at the beginning of the epoch [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:40:42.054.713 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:40:42.054.780 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:40:42.054.817 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_1 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.055.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.056.735 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1037 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.069.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 1, total running count: 937 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.070.861 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.072.776 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1038 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:42.085.465 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 2, total running count: 938 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:42.086.811 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.088.492 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1039 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:42.101.343 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 3, total running count: 939 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:42.102.511 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.104.117 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1040 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.117.031 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 4, total running count: 940 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.118.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.119.821 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1041 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.132.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 5, total running count: 941 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.133.777 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.135.796 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1042 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:42.148.470 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 6, total running count: 942 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.149.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.151.725 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1043 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:42.163.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 7, total running count: 943 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:42.164.812 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.167.412 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1044 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:42.179.371 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 8, total running count: 944 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:42.180.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.183.068 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1045 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.194.971 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 9, total running count: 945 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.195.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.197.927 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1046 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:42.210.483 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 10, total running count: 946 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:42.211.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.213.789 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1047 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.225.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 11, total running count: 947 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:42.226.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.229.469 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1048 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:42.241.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 12, total running count: 948 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.242.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.244.306 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1049 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:42.257.116 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 13, total running count: 949 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:42.258.481 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.260.049 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1050 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.272.960 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 14, total running count: 950 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.274.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.276.753 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1051 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.288.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 15, total running count: 951 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:42.289.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.292.276 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1052 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.304.135 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 16, total running count: 952 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.305.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.306.809 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1053 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:42.319.608 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 17, total running count: 953 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.320.370 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.322.718 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1054 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.334.841 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 18, total running count: 954 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.335.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.338.270 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1055 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.350.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 19, total running count: 955 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.351.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.353.785 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1056 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.365.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 20, total running count: 956 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.366.969 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.369.631 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1057 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.381.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 21, total running count: 957 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.382.439 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.384.176 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1058 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.396.949 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 22, total running count: 958 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:42.397.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.399.876 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1059 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.412.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 23, total running count: 959 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:42.413.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.415.379 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1060 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.427.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 24, total running count: 960 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.428.998 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.430.881 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1061 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.443.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 25, total running count: 961 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.444.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.446.456 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1062 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.458.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 26, total running count: 962 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:42.460.191 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.462.264 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1063 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.474.731 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 27, total running count: 963 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.475.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.478.092 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1064 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.489.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 28, total running count: 964 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:42.491.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.492.698 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1065 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.505.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 29, total running count: 965 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.506.485 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.508.286 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1066 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.520.992 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 30, total running count: 966 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.522.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.524.125 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1067 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:42.536.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 31, total running count: 967 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.537.344 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.539.841 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1068 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:42.551.966 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 32, total running count: 968 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:42.552.793 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.555.254 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1069 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.567.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 33, total running count: 969 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.568.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.570.848 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1070 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.582.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 34, total running count: 970 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.583.736 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.586.234 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1071 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.598.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 35, total running count: 971 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.599.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.601.780 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1072 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.613.660 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 36, total running count: 972 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.614.725 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.617.345 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1073 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:42.629.120 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 37, total running count: 973 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.630.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.632.030 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1074 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:42.644.695 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 38, total running count: 974 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:42.645.678 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.647.862 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1075 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.660.586 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 39, total running count: 975 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.660.966 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.663.500 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1076 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.675.592 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 40, total running count: 976 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.676.647 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.678.902 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1077 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.690.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 41, total running count: 977 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.692.058 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.694.515 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1078 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:42.706.596 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 42, total running count: 978 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.707.601 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.710.174 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1079 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:42.722.295 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 43, total running count: 979 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.723.151 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.725.675 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1080 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.737.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 44, total running count: 980 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.738.689 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.740.462 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1081 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:42.753.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 45, total running count: 981 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:42.754.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.756.154 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1082 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:42.768.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 46, total running count: 982 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.769.644 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.772.066 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1083 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.784.138 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 47, total running count: 983 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.785.002 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.786.673 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1084 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:42.799.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 48, total running count: 984 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.800.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.802.165 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1085 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.815.141 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 49, total running count: 985 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.815.863 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.817.652 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1086 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:42.830.438 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 50, total running count: 986 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.831.259 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.832.957 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1087 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.845.615 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 51, total running count: 987 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:42.846.547 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.848.733 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1088 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.860.839 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 52, total running count: 988 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:42.861.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.864.348 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1089 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:42.876.437 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 53, total running count: 989 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.877.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.879.077 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1090 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.891.670 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 54, total running count: 990 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.892.776 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.894.609 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1091 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.907.213 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 55, total running count: 991 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:42.908.400 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.910.153 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1092 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:42.922.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 56, total running count: 992 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.923.988 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.925.604 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1093 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.938.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 57, total running count: 993 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.939.563 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.942.295 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1094 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.954.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 58, total running count: 994 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.955.068 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.956.599 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1095 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.969.596 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 59, total running count: 995 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:42.970.521 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.973.149 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1096 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:42.985.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 60, total running count: 996 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:42.985.889 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:42.987.509 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1097 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.000.222 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 61, total running count: 997 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.001.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.003.085 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1098 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.016.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 62, total running count: 998 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.017.182 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.020.003 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1099 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.033.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 63, total running count: 999 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.034.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.036.832 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1100 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.049.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 64, total running count: 1000 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:43.049.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.051.309 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1101 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:43.064.051 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 65, total running count: 1001 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.064.425 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.067.029 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1102 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.078.967 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 66, total running count: 1002 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:43.079.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.081.674 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1103 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.094.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 67, total running count: 1003 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.094.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.096.147 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1104 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.108.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 68, total running count: 1004 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:43.109.805 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.111.665 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1105 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.124.233 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 69, total running count: 1005 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.125.208 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.127.380 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1106 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.139.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 70, total running count: 1006 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.141.080 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.143.008 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1107 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:43.155.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 71, total running count: 1007 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:43.156.501 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.158.788 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1108 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.171.027 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 72, total running count: 1008 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:43.172.016 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.174.264 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1109 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.186.480 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 73, total running count: 1009 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.187.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.189.786 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1110 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.201.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 74, total running count: 1010 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.202.789 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.205.279 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1111 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:43.216.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 75, total running count: 1011 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.218.380 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.220.811 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1112 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.232.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 76, total running count: 1012 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:43.233.832 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.236.524 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1113 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.248.235 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 77, total running count: 1013 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:43.249.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.251.216 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1114 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.264.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 78, total running count: 1014 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.264.880 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.266.824 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1115 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.279.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 79, total running count: 1015 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:43.280.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.282.812 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1116 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:43.294.755 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 80, total running count: 1016 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.295.812 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.298.531 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1117 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:43.310.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 81, total running count: 1017 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:43.311.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.313.363 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1118 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.325.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 82, total running count: 1018 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.327.039 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.329.036 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1119 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:43.341.468 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 83, total running count: 1019 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.342.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.344.607 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1120 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.356.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 84, total running count: 1020 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.357.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.360.175 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1121 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.372.311 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 85, total running count: 1021 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.373.044 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.375.774 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1122 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.387.216 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 86, total running count: 1022 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.388.694 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.390.380 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1123 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.403.037 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 87, total running count: 1023 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.404.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.406.174 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1124 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.418.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 88, total running count: 1024 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:43.419.926 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.421.545 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1125 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.434.645 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 89, total running count: 1025 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.435.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.437.232 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1126 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.449.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 90, total running count: 1026 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:43.451.065 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.452.736 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1127 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.465.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 91, total running count: 1027 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.466.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.468.219 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1128 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.480.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 92, total running count: 1028 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.481.961 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.483.776 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1129 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.496.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 93, total running count: 1029 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:43.497.530 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.499.517 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1130 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.511.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 94, total running count: 1030 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.513.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.515.340 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1131 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.527.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 95, total running count: 1031 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.529.590 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.531.305 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1132 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:43.543.943 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 96, total running count: 1032 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.545.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.546.863 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1133 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.559.486 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 97, total running count: 1033 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:43.560.259 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.562.375 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1134 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:43.574.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 98, total running count: 1034 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.575.583 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.578.016 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1135 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:43.589.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 99, total running count: 1035 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:43.590.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.592.589 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1136 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:43.605.203 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 100, total running count: 1036 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.606.255 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.608.263 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1137 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.620.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 101, total running count: 1037 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.622.031 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.623.897 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1138 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.636.424 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 102, total running count: 1038 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.637.563 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.639.395 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1139 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.651.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 103, total running count: 1039 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:43.652.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.654.977 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1140 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:43.667.156 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 104, total running count: 1040 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.668.738 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.670.522 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1141 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.683.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 105, total running count: 1041 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:43.684.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.686.315 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1142 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.698.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 106, total running count: 1042 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.699.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.702.106 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1143 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.714.444 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 107, total running count: 1043 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.715.444 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.717.792 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1144 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.730.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 108, total running count: 1044 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.731.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.733.285 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1145 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.745.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 109, total running count: 1045 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:43.746.385 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.748.758 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1146 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.760.683 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 110, total running count: 1046 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.761.839 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.764.135 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1147 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.776.343 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 111, total running count: 1047 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.777.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.779.740 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1148 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.791.713 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 112, total running count: 1048 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.792.725 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.794.389 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1149 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.807.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 113, total running count: 1049 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.808.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.810.187 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1150 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.822.638 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 114, total running count: 1050 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.823.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.825.678 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1151 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.837.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 115, total running count: 1051 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.838.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.841.253 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1152 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:43.853.379 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 116, total running count: 1052 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:43.854.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.856.658 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1153 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.868.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 117, total running count: 1053 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.869.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.872.258 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1154 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:43.884.181 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 118, total running count: 1054 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.885.217 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.886.998 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1155 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.899.727 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 119, total running count: 1055 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.900.483 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.902.838 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1156 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.914.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 120, total running count: 1056 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.915.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.918.410 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1157 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.930.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 121, total running count: 1057 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:43.931.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.933.868 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1158 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.945.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 122, total running count: 1058 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.948.872 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.950.615 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1159 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:43.963.163 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 123, total running count: 1059 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:43.965.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.967.681 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1160 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:43.979.856 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 124, total running count: 1060 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:43.980.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.983.447 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1161 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:43.995.287 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 125, total running count: 1061 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:43.996.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:43.998.994 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1162 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.010.860 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 126, total running count: 1062 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.011.930 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.014.583 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1163 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.026.475 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 127, total running count: 1063 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.027.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.029.336 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1164 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.042.024 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 128, total running count: 1064 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.042.984 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.045.187 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1165 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.057.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 129, total running count: 1065 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.058.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.060.667 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1166 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.072.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 130, total running count: 1066 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.073.915 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.076.093 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1167 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.088.277 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 131, total running count: 1067 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.089.091 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.091.618 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1168 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.103.560 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 132, total running count: 1068 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.104.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.107.350 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1169 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.119.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 133, total running count: 1069 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.120.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.122.099 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1170 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.134.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 134, total running count: 1070 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.135.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.137.792 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1171 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.150.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 135, total running count: 1071 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:44.151.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.153.634 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1172 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:44.165.361 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 136, total running count: 1072 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.166.508 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.168.211 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1173 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.180.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 137, total running count: 1073 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:44.182.001 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.183.772 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1174 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.196.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 138, total running count: 1074 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.197.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.199.472 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1175 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.211.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 139, total running count: 1075 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.212.596 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.214.188 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1176 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.227.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 140, total running count: 1076 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.227.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.229.799 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1177 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:44.242.399 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 141, total running count: 1077 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.243.403 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.245.581 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1178 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.257.779 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 142, total running count: 1078 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.259.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.261.394 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1179 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.273.880 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 143, total running count: 1079 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.275.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.277.233 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1180 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:44.289.608 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 144, total running count: 1080 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.290.804 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.292.995 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1181 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.305.403 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 145, total running count: 1081 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.306.421 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.308.736 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1182 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.320.790 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 146, total running count: 1082 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.321.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.324.363 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1183 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.336.288 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 147, total running count: 1083 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.337.224 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.339.874 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1184 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.351.629 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 148, total running count: 1084 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.352.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.354.343 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1185 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.367.195 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 149, total running count: 1085 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.368.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.370.177 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1186 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.382.437 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 150, total running count: 1086 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.383.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.386.104 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1187 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.397.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 151, total running count: 1087 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.398.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.401.644 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1188 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:44.413.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 152, total running count: 1088 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.416.066 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.418.575 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1189 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:44.430.711 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 153, total running count: 1089 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.432.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.434.564 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1190 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.446.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 154, total running count: 1090 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.447.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.450.253 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1191 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.462.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 155, total running count: 1091 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.463.627 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.465.761 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1192 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.477.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 156, total running count: 1092 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.479.217 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.481.323 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1193 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.493.791 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 157, total running count: 1093 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.494.967 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.497.422 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1194 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.509.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 158, total running count: 1094 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.510.385 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.512.116 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1195 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.524.791 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 159, total running count: 1095 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.525.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.527.744 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1196 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.540.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 160, total running count: 1096 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:44.541.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.543.579 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1197 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.555.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 161, total running count: 1097 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.560.859 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.562.969 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1198 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.575.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 162, total running count: 1098 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.576.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.578.552 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1199 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.590.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 163, total running count: 1099 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.592.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.594.171 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1200 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.606.592 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 164, total running count: 1100 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.607.527 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.609.969 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1201 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.621.937 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 165, total running count: 1101 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.623.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.625.740 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1202 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.637.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 166, total running count: 1102 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.638.627 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.641.203 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1203 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.653.121 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 167, total running count: 1103 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.654.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.656.689 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1204 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.668.521 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 168, total running count: 1104 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.669.539 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.672.160 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1205 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.684.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 169, total running count: 1105 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.684.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.686.555 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1206 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.699.213 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 170, total running count: 1106 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.700.262 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.702.353 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1207 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.714.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 171, total running count: 1107 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.719.195 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.721.749 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1208 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.733.552 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 172, total running count: 1108 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.734.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.737.467 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1209 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.749.198 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 173, total running count: 1109 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.750.601 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.752.979 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1210 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.765.002 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 174, total running count: 1110 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.766.053 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.767.778 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1211 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.780.434 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 175, total running count: 1111 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.781.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.783.393 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1212 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.795.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 176, total running count: 1112 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.796.769 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.798.975 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1213 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.811.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 177, total running count: 1113 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.812.262 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.814.499 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1214 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.826.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 178, total running count: 1114 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.829.361 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.831.435 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1215 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.843.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 179, total running count: 1115 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.845.897 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.848.504 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1216 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.860.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 180, total running count: 1116 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.861.770 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.864.134 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1217 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.876.183 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 181, total running count: 1117 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.877.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.879.743 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1218 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.891.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 182, total running count: 1118 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.893.287 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.895.452 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1219 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.907.692 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 183, total running count: 1119 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.908.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.911.350 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1220 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:44.923.174 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 184, total running count: 1120 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.926.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.928.074 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1221 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.940.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 185, total running count: 1121 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:44.944.203 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.945.954 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1222 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:44.958.691 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 186, total running count: 1122 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:44.959.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.961.736 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1223 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.974.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 187, total running count: 1123 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:44.975.511 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.977.509 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1224 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:44.989.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 188, total running count: 1124 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:44.991.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:44.993.201 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1225 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.005.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 189, total running count: 1125 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.006.945 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.008.986 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1226 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.021.459 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 190, total running count: 1126 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.022.358 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.024.747 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1227 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.036.733 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 191, total running count: 1127 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.040.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.042.795 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1228 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:45.054.556 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 192, total running count: 1128 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.055.746 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.058.220 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1229 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.070.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 193, total running count: 1129 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.071.460 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.073.677 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1230 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.085.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 194, total running count: 1130 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.086.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.089.554 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1231 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.100.982 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 195, total running count: 1131 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.102.148 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.104.139 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1232 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.116.421 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 196, total running count: 1132 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.117.737 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.119.800 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1233 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.132.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 197, total running count: 1133 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.135.441 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.137.797 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1234 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.149.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 198, total running count: 1134 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.152.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.154.601 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1235 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:45.166.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 199, total running count: 1135 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:45.168.020 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.170.559 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1236 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.182.516 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 200, total running count: 1136 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.184.030 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.186.447 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1237 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.198.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 201, total running count: 1137 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.199.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.201.921 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1238 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.214.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 202, total running count: 1138 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.215.688 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.217.786 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1239 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:45.230.061 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 203, total running count: 1139 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.231.345 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.233.870 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1240 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:45.245.832 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 204, total running count: 1140 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.253.421 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.255.144 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1241 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.267.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 205, total running count: 1141 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.271.046 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.273.280 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1242 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.285.295 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 206, total running count: 1142 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.287.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.289.091 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1243 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.301.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 207, total running count: 1143 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.302.915 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.304.615 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1244 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.317.199 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 208, total running count: 1144 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.318.885 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.321.239 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1245 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.333.262 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 209, total running count: 1145 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.334.939 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.337.034 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1246 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.349.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 210, total running count: 1146 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.351.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.352.870 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1247 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.365.629 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 211, total running count: 1147 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.367.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.369.776 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1248 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.381.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 212, total running count: 1148 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.383.068 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.385.410 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1249 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.397.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 213, total running count: 1149 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.398.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.400.895 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1250 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.413.173 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 214, total running count: 1150 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.414.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.416.533 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1251 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.428.696 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 215, total running count: 1151 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:45.430.249 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.432.417 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1252 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.444.682 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 216, total running count: 1152 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.446.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.448.065 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1253 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.460.758 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 217, total running count: 1153 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:45.462.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.464.744 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1254 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:45.477.118 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 218, total running count: 1154 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:45.478.380 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.480.178 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1255 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.492.840 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 219, total running count: 1155 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.494.073 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.495.729 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1256 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.508.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 220, total running count: 1156 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.509.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.511.523 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1257 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.524.145 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 221, total running count: 1157 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.525.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.527.406 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1258 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.539.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 222, total running count: 1158 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.541.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.543.956 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1259 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.555.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 223, total running count: 1159 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.557.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.559.518 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1260 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.571.511 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 224, total running count: 1160 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.573.173 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.575.285 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1261 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:45.587.624 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 225, total running count: 1161 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.588.859 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.591.094 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1262 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.603.252 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 226, total running count: 1162 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.604.624 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.606.897 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1263 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.618.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 227, total running count: 1163 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.620.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.622.405 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1264 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.634.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 228, total running count: 1164 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.635.733 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.637.833 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1265 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.650.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 229, total running count: 1165 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.651.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.653.347 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1266 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.665.740 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 230, total running count: 1166 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.666.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.669.053 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1267 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.681.187 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 231, total running count: 1167 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.682.602 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.684.305 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1268 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.697.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 232, total running count: 1168 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.698.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.699.922 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1269 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.712.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 233, total running count: 1169 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.714.080 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.715.729 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1270 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.728.412 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 234, total running count: 1170 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.729.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.731.390 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1271 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.743.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 235, total running count: 1171 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:45.745.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.746.970 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1272 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.759.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 236, total running count: 1172 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.760.856 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.762.614 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1273 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.775.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 237, total running count: 1173 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.776.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.778.511 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1274 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.790.709 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 238, total running count: 1174 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.792.140 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.794.128 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1275 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.806.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 239, total running count: 1175 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.807.614 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.809.903 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1276 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.822.145 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 240, total running count: 1176 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.823.241 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.825.596 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1277 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.837.602 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 241, total running count: 1177 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.838.724 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.841.421 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1278 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.852.942 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 242, total running count: 1178 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.854.380 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.856.112 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1279 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.868.693 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 243, total running count: 1179 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.869.905 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.871.834 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1280 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.884.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 244, total running count: 1180 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:45.885.243 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.887.594 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1281 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:45.899.833 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 245, total running count: 1181 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.900.705 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.903.025 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1282 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:45.914.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 246, total running count: 1182 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.916.460 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.918.769 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1283 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:45.930.909 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 247, total running count: 1183 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.931.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.934.367 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1284 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.946.458 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 248, total running count: 1184 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.947.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.949.854 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1285 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.961.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 249, total running count: 1185 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:45.963.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.965.606 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1286 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:45.977.575 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 250, total running count: 1186 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:45.978.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.980.132 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1287 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.992.992 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 251, total running count: 1187 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:45.994.164 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:45.996.778 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1288 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.008.370 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 252, total running count: 1188 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.009.858 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.012.235 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1289 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.024.269 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 253, total running count: 1189 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:46.025.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.027.991 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1290 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.040.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 254, total running count: 1190 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.040.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.043.584 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1291 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.055.400 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 255, total running count: 1191 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.056.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.058.324 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1292 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.070.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 256, total running count: 1192 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.071.808 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.074.004 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1293 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.086.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 257, total running count: 1193 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.087.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.089.598 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1294 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.101.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 258, total running count: 1194 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.103.404 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.105.597 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1295 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.117.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 259, total running count: 1195 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:46.119.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.121.374 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1296 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.133.622 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 260, total running count: 1196 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.134.544 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.137.448 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1297 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.148.856 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 261, total running count: 1197 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.149.998 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.152.043 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1298 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.164.246 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 262, total running count: 1198 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.165.292 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.167.743 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1299 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.179.675 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 263, total running count: 1199 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.180.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.182.515 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1300 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.195.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 264, total running count: 1200 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.196.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.198.178 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1301 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.210.820 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 265, total running count: 1201 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:46.212.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.214.147 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1302 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.226.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 266, total running count: 1202 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.227.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.229.968 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1303 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.241.944 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 267, total running count: 1203 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:46.242.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.244.439 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1304 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.257.200 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 268, total running count: 1204 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.257.947 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.259.899 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1305 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.272.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 269, total running count: 1205 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.273.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.275.344 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1306 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.288.077 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 270, total running count: 1206 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.288.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.290.813 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1307 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.303.473 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 271, total running count: 1207 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.304.613 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.306.354 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1308 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.318.899 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 272, total running count: 1208 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.319.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.321.802 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1309 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.334.627 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 273, total running count: 1209 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:46.335.725 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.338.423 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1310 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.349.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 274, total running count: 1210 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.351.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.353.833 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1311 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:46.365.805 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 275, total running count: 1211 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.366.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.369.357 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1312 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.381.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 276, total running count: 1212 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.382.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.384.777 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1313 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.396.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 277, total running count: 1213 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.398.035 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.400.263 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1314 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:46.412.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 278, total running count: 1214 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.413.255 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.414.862 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1315 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.427.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 279, total running count: 1215 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.428.732 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.430.743 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1316 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.443.028 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 280, total running count: 1216 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:46.444.163 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.446.322 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1317 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.458.689 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 281, total running count: 1217 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.459.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.461.841 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1318 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.474.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 282, total running count: 1218 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.475.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.477.440 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1319 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.489.778 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 283, total running count: 1219 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.490.769 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.493.268 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1320 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.505.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 284, total running count: 1220 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.506.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.508.971 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1321 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.520.989 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 285, total running count: 1221 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.522.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.524.920 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1322 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.536.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 286, total running count: 1222 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.537.831 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.539.593 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1323 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.552.177 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 287, total running count: 1223 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.553.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.554.985 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1324 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.567.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 288, total running count: 1224 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.568.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.570.505 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1325 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:46.582.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 289, total running count: 1225 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.584.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.586.253 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1326 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.598.619 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 290, total running count: 1226 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.600.065 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.602.120 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1327 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.614.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 291, total running count: 1227 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.615.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.617.597 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1328 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.630.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 292, total running count: 1228 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.630.854 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.633.048 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1329 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.645.382 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 293, total running count: 1229 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.646.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.648.641 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1330 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.660.805 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 294, total running count: 1230 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.661.788 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.663.384 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1331 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.676.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 295, total running count: 1231 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.677.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.678.928 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1332 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.691.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 296, total running count: 1232 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.692.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.694.593 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1333 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.707.286 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 297, total running count: 1233 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.708.407 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.710.174 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1334 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.722.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 298, total running count: 1234 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:46.723.771 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.725.762 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1335 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.738.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 299, total running count: 1235 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.739.213 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.741.421 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1336 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.756.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 300, total running count: 1236 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.756.691 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.758.614 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1337 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.772.018 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 301, total running count: 1237 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.772.389 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.774.624 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1338 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.786.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 302, total running count: 1238 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:46.787.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.789.575 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1339 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.801.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 303, total running count: 1239 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.802.053 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.804.113 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1340 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.816.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 304, total running count: 1240 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.817.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.820.150 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1341 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.831.808 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 305, total running count: 1241 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.832.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.834.598 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1342 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.847.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 306, total running count: 1242 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.848.335 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.850.235 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1343 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:46.862.615 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 307, total running count: 1243 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.864.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.866.464 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1344 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.878.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 308, total running count: 1244 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.879.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.881.353 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1345 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.894.150 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 309, total running count: 1245 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.895.207 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.897.437 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1346 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.909.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 310, total running count: 1246 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.910.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.912.539 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1347 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.925.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 311, total running count: 1247 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.926.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.928.241 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1348 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.940.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 312, total running count: 1248 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.941.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.943.979 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1349 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:46.956.633 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 313, total running count: 1249 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.957.476 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.959.773 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1350 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:46.971.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 314, total running count: 1250 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:46.973.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.976.000 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1351 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:46.987.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 315, total running count: 1251 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:46.988.590 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:46.990.573 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1352 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:47.003.019 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 316, total running count: 1252 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.004.118 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.006.693 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1353 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.018.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 317, total running count: 1253 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.019.987 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.022.741 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1354 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.034.515 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 318, total running count: 1254 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.035.686 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.037.595 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1355 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.050.091 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 319, total running count: 1255 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.051.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.053.649 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1356 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.065.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 320, total running count: 1256 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:47.066.722 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.069.457 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1357 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.081.466 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 321, total running count: 1257 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:47.082.335 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.084.161 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1358 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.096.693 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 322, total running count: 1258 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.098.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.100.063 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1359 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:47.112.567 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 323, total running count: 1259 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.113.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.115.994 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1360 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:47.128.168 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 324, total running count: 1260 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.129.246 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.132.027 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1361 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.143.840 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 325, total running count: 1261 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.144.945 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.146.795 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1362 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:47.159.658 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 326, total running count: 1262 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.160.793 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.163.053 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1363 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.175.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 327, total running count: 1263 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:47.176.695 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.179.085 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1364 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.191.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 328, total running count: 1264 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.192.425 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.195.018 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1365 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.207.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 329, total running count: 1265 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.209.727 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.212.338 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1366 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.224.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 330, total running count: 1266 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.225.632 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.228.428 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1367 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.240.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 331, total running count: 1267 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.241.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.244.376 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1368 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.256.527 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 332, total running count: 1268 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.257.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.259.101 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1369 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:47.271.951 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 333, total running count: 1269 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:47.273.362 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.276.191 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1370 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.287.987 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 334, total running count: 1270 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.289.326 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.291.865 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1371 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.303.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 335, total running count: 1271 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.305.086 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.307.751 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1372 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.319.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 336, total running count: 1272 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.320.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.322.833 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1373 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:47.335.248 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 337, total running count: 1273 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.336.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.338.659 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1374 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.351.043 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 338, total running count: 1274 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.351.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.354.364 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1375 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.366.700 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 339, total running count: 1275 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.367.742 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.369.783 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1376 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.382.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 340, total running count: 1276 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.383.335 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.385.346 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1377 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.398.007 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 341, total running count: 1277 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.399.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.401.253 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1378 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:47.413.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 342, total running count: 1278 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.414.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.416.646 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1379 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:47.429.523 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 343, total running count: 1279 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.430.674 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.432.430 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1380 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.445.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 344, total running count: 1280 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:47.446.338 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.448.196 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1381 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:47.461.122 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 345, total running count: 1281 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.461.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.463.805 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1382 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.476.694 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 346, total running count: 1282 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.478.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.480.599 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1383 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.492.702 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 347, total running count: 1283 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.493.678 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.496.038 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1384 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.508.688 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 348, total running count: 1284 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.509.474 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.511.554 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1385 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:47.524.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 349, total running count: 1285 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.525.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.527.443 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1386 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:47.539.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 350, total running count: 1286 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.542.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.544.156 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1387 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.557.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 351, total running count: 1287 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:47.557.875 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.560.053 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1388 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.572.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 352, total running count: 1288 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.573.295 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:47.588.339 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 353, total running count: 1289 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.588.724 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.589.792 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1389 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.596.643 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1390 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:47.603.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 354, total running count: 1290 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:47.604.264 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.606.572 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1391 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.619.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 355, total running count: 1291 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.619.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.622.214 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1392 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.634.545 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 356, total running count: 1292 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.635.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.637.614 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1393 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.650.636 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 357, total running count: 1293 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:47.651.267 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.653.206 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1394 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.668.489 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 358, total running count: 1294 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:47.668.882 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.671.076 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1395 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:47.684.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 359, total running count: 1295 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.684.792 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.686.489 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1396 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.700.124 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 360, total running count: 1296 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.700.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.702.357 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1397 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:47.715.592 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 361, total running count: 1297 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:47.715.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.718.048 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1398 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:47.731.153 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 362, total running count: 1298 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.731.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.733.592 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1399 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.746.762 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 363, total running count: 1299 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.747.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.749.040 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1400 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.762.541 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 364, total running count: 1300 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.762.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.764.665 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1401 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.777.727 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 365, total running count: 1301 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.778.100 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.780.498 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1402 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:47.792.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 366, total running count: 1302 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.793.212 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.795.141 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1403 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.808.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 367, total running count: 1303 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.808.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.810.834 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1404 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.823.354 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 368, total running count: 1304 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.823.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.826.640 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1405 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.838.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 369, total running count: 1305 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.839.033 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.840.975 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1406 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.853.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 370, total running count: 1306 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.854.816 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.856.572 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1407 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.869.665 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 371, total running count: 1307 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.870.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.872.202 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1408 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.885.453 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 372, total running count: 1308 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.886.318 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.889.050 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1409 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.901.092 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 373, total running count: 1309 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.902.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.904.730 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1410 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.916.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 374, total running count: 1310 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:47.917.554 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.919.249 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1411 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:47.932.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 375, total running count: 1311 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.933.156 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.935.233 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1412 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.947.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 376, total running count: 1312 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:47.948.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.950.963 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1413 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:47.963.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 377, total running count: 1313 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.964.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.966.512 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1414 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:47.979.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 378, total running count: 1314 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.980.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.982.253 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1415 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:47.994.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 379, total running count: 1315 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:47.995.597 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:47.998.167 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1416 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.010.587 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 380, total running count: 1316 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.011.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.013.823 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1417 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.030.713 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 381, total running count: 1317 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.031.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.032.815 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1418 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.047.037 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 382, total running count: 1318 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.047.398 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.049.812 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1419 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.062.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 383, total running count: 1319 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:48.062.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.065.332 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1420 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.078.024 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 384, total running count: 1320 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.078.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.080.787 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1421 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.093.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 385, total running count: 1321 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.093.721 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.096.500 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1422 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:48.108.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 386, total running count: 1322 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.108.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.111.025 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1423 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.124.011 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 387, total running count: 1323 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:48.124.388 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.126.738 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1424 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.139.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 388, total running count: 1324 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:48.139.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.141.537 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1425 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.155.058 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 389, total running count: 1325 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:48.155.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.157.113 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1426 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.170.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 390, total running count: 1326 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.170.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.172.878 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1427 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.185.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 391, total running count: 1327 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.186.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.188.529 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1428 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.201.194 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 392, total running count: 1328 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.202.087 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.204.108 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1429 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.217.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 393, total running count: 1329 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.217.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.219.819 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1430 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.232.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 394, total running count: 1330 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.233.277 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.235.668 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1431 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.248.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 395, total running count: 1331 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.249.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.251.549 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1432 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.264.267 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 396, total running count: 1332 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.264.878 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.267.373 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1433 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.279.783 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 397, total running count: 1333 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.280.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.283.083 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1434 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.295.278 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 398, total running count: 1334 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.296.016 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.297.858 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1435 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:48.311.184 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 399, total running count: 1335 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.311.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.313.608 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1436 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.326.733 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 400, total running count: 1336 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.327.254 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.329.362 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1437 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.342.035 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 401, total running count: 1337 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.342.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.345.284 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1438 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.357.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 402, total running count: 1338 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.358.668 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.361.328 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1439 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.373.424 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 403, total running count: 1339 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.374.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.376.097 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1440 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.389.782 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 404, total running count: 1340 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.390.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.391.914 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1441 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.405.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 405, total running count: 1341 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.405.789 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.407.776 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1442 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.420.660 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 406, total running count: 1342 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.421.202 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.423.364 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1443 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.436.155 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 407, total running count: 1343 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.436.775 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.438.749 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1444 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.451.832 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 408, total running count: 1344 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:48.452.262 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.454.203 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1445 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:48.467.264 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 409, total running count: 1345 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.467.851 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.469.836 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1446 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.482.728 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 410, total running count: 1346 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.483.455 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.485.416 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1447 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.498.269 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 411, total running count: 1347 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.499.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.501.273 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1448 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:48.513.820 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 412, total running count: 1348 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:48.514.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.517.288 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1449 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:48.529.403 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 413, total running count: 1349 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.530.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.532.878 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1450 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.544.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 414, total running count: 1350 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.545.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.548.450 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1451 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.560.460 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 415, total running count: 1351 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.561.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.562.993 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1452 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.576.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 416, total running count: 1352 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.577.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.578.845 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1453 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.591.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 417, total running count: 1353 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.592.657 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.594.780 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1454 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.607.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 418, total running count: 1354 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.608.191 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.610.325 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1455 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.623.317 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 419, total running count: 1355 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.623.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.626.221 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1456 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.638.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 420, total running count: 1356 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.639.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.642.211 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1457 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.654.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 421, total running count: 1357 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.655.080 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.656.685 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1458 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.669.860 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 422, total running count: 1358 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.670.495 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.672.273 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1459 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.685.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 423, total running count: 1359 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.686.233 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.687.824 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1460 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.701.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 424, total running count: 1360 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.701.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.703.557 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1461 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.716.809 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 425, total running count: 1361 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.717.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.718.965 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1462 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.731.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 426, total running count: 1362 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.732.780 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.734.591 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1463 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:48.747.589 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 427, total running count: 1363 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.748.291 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.750.215 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1464 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.763.226 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 428, total running count: 1364 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.763.982 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.765.854 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1465 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.778.858 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 429, total running count: 1365 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.779.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.781.381 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1466 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.794.391 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 430, total running count: 1366 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.795.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.798.685 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1467 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.811.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 431, total running count: 1367 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.811.720 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.814.370 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1468 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.826.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 432, total running count: 1368 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.827.164 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.828.982 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1469 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.841.938 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 433, total running count: 1369 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:48.842.336 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.844.589 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1470 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:48.856.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 434, total running count: 1370 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.858.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.860.137 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1471 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.872.665 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 435, total running count: 1371 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.873.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.875.516 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1472 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.888.330 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 436, total running count: 1372 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:48.889.162 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.891.454 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1473 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.903.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 437, total running count: 1373 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:48.904.788 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.907.393 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1474 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.919.688 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 438, total running count: 1374 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.920.415 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.923.001 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1475 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.935.145 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 439, total running count: 1375 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.935.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.937.623 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1476 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.950.594 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 440, total running count: 1376 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.951.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.953.788 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1477 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.966.141 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 441, total running count: 1377 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.966.947 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.969.376 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1478 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:48.981.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 442, total running count: 1378 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:48.982.402 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.984.979 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1479 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:48.997.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 443, total running count: 1379 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:48.997.909 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:48.999.797 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1480 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.012.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 444, total running count: 1380 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.013.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.015.737 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1481 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.028.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 445, total running count: 1381 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.028.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.031.270 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1482 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:49.043.174 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 446, total running count: 1382 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:49.044.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.046.812 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1483 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:49.058.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 447, total running count: 1383 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.059.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.061.294 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1484 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.074.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 448, total running count: 1384 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.075.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.076.989 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1485 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.089.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 449, total running count: 1385 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.090.474 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.092.925 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1486 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:49.105.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 450, total running count: 1386 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.105.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.107.573 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1487 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.120.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 451, total running count: 1387 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.121.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.123.330 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1488 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.136.063 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 452, total running count: 1388 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.136.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.139.137 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1489 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.151.440 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 453, total running count: 1389 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.152.182 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.154.778 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1490 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.166.640 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 454, total running count: 1390 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.167.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.170.232 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1491 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.182.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 455, total running count: 1391 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:49.182.982 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.184.665 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1492 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.197.566 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 456, total running count: 1392 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:49.198.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.200.244 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1493 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:49.212.693 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 457, total running count: 1393 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.213.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.216.243 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1494 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:49.228.382 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 458, total running count: 1394 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.229.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.231.006 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1495 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.243.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 459, total running count: 1395 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.245.056 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.246.854 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1496 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.259.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 460, total running count: 1396 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.261.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.263.672 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1497 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:49.275.935 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 461, total running count: 1397 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.276.769 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.278.372 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1498 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.291.345 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 462, total running count: 1398 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:49.292.175 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.294.216 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1499 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.306.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 463, total running count: 1399 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.307.719 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.309.936 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1500 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:49.322.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 464, total running count: 1400 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:49.323.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.325.542 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1501 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:49.337.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 465, total running count: 1401 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.338.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.341.433 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1502 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.353.250 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 466, total running count: 1402 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.354.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.356.033 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1503 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.368.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 467, total running count: 1403 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.369.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.371.760 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1504 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:49.384.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 468, total running count: 1404 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:40:49.385.620 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_1 execution count: 3, execution time: 7330.67 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:40:49.385.855 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_1 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:40:49.385.962 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty epoch: 3 step: 468, loss is 2.2889134883880615 [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:40:49.386.996 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.h:76] ContinueSend] continue send at the beginning of the epoch [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:40:49.388.143 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:40:49.388.206 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:40:49.388.242 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_1 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.388.606 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.391.073 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1505 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.403.124 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 1, total running count: 1405 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:49.404.593 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.406.920 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1506 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:49.419.140 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 2, total running count: 1406 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.420.265 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.422.414 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1507 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.434.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 3, total running count: 1407 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.435.992 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.438.099 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1508 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.450.698 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 4, total running count: 1408 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.451.745 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.453.604 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1509 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:49.466.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 5, total running count: 1409 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.467.199 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.469.559 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1510 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.481.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 6, total running count: 1410 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.482.715 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.485.292 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1511 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:49.497.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 7, total running count: 1411 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.498.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.500.891 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1512 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.512.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 8, total running count: 1412 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.514.370 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.516.691 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1513 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.528.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 9, total running count: 1413 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.530.164 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.532.722 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1514 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.544.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 10, total running count: 1414 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:49.545.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.547.375 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1515 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:49.560.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 11, total running count: 1415 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.561.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.564.072 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1516 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:49.576.035 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 12, total running count: 1416 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.576.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.579.699 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1517 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.591.371 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 13, total running count: 1417 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.592.382 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.594.387 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1518 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:49.606.838 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 14, total running count: 1418 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:49.607.848 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.610.284 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1519 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:49.622.630 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 15, total running count: 1419 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:49.623.350 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.626.023 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1520 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.637.944 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 16, total running count: 1420 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.638.841 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.640.456 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1521 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.653.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 17, total running count: 1421 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.654.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.656.235 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1522 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:49.668.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 18, total running count: 1422 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.669.725 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.671.956 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1523 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.684.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 19, total running count: 1423 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.685.207 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.687.577 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1524 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.699.622 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 20, total running count: 1424 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.700.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.703.227 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1525 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.715.205 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 21, total running count: 1425 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.716.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.718.064 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1526 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.730.853 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 22, total running count: 1426 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.731.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.733.736 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1527 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.746.122 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 23, total running count: 1427 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.747.186 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.749.428 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1528 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:49.761.583 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 24, total running count: 1428 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.762.764 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.765.291 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1529 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:49.777.009 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 25, total running count: 1429 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.778.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.779.890 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1530 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.792.694 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 26, total running count: 1430 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:49.793.781 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.795.458 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1531 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:49.808.184 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 27, total running count: 1431 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.809.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.811.019 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1532 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.823.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 28, total running count: 1432 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.824.638 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.826.550 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1533 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.839.059 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 29, total running count: 1433 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.839.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.842.487 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1534 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.854.313 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 30, total running count: 1434 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.855.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.857.003 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1535 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.869.445 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 31, total running count: 1435 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.870.468 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.872.445 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1536 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:49.884.997 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 32, total running count: 1436 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:49.885.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.887.851 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1537 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.900.362 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 33, total running count: 1437 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.901.370 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.903.291 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1538 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.915.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 34, total running count: 1438 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.916.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.918.763 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1539 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.930.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 35, total running count: 1439 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.932.084 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.934.263 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1540 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.946.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 36, total running count: 1440 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:49.947.633 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.949.641 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1541 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.962.061 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 37, total running count: 1441 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:49.962.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.965.268 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1542 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.977.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 38, total running count: 1442 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.978.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.980.930 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1543 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:49.992.905 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 39, total running count: 1443 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:49.993.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:49.995.505 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1544 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.008.343 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 40, total running count: 1444 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.009.168 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.011.208 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1545 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.023.704 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 41, total running count: 1445 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.024.734 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.026.789 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1546 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.039.147 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 42, total running count: 1446 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.040.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.042.316 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1547 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.054.882 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 43, total running count: 1447 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.055.636 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.057.850 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1548 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.070.151 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 44, total running count: 1448 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.071.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.073.753 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1549 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.085.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 45, total running count: 1449 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.086.691 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.089.378 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1550 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:50.100.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 46, total running count: 1450 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.102.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.103.993 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1551 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.116.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 47, total running count: 1451 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.117.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.119.403 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1552 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.131.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 48, total running count: 1452 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:50.132.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.134.881 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1553 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.147.248 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 49, total running count: 1453 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:50.148.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.150.340 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1554 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.162.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 50, total running count: 1454 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.163.885 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.165.923 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1555 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.178.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 51, total running count: 1455 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.179.444 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.181.597 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1556 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:50.194.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 52, total running count: 1456 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.195.140 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.197.292 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1557 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.209.632 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 53, total running count: 1457 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.210.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.213.074 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1558 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:50.225.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 54, total running count: 1458 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.225.985 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.228.609 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1559 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.240.437 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 55, total running count: 1459 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:50.241.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.243.988 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1560 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.256.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 56, total running count: 1460 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.256.728 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.258.319 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1561 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.271.042 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 57, total running count: 1461 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.272.210 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.273.928 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1562 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:50.286.552 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 58, total running count: 1462 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.287.645 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.289.558 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1563 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.302.132 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 59, total running count: 1463 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.303.121 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.304.944 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1564 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.317.540 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 60, total running count: 1464 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.318.757 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.320.575 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1565 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.333.020 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 61, total running count: 1465 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.334.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.335.995 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1566 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:50.348.851 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 62, total running count: 1466 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.349.752 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.351.580 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1567 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.364.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 63, total running count: 1467 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:50.365.196 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.367.475 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1568 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.379.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 64, total running count: 1468 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:50.380.815 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.383.364 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1569 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:50.395.370 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 65, total running count: 1469 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.396.341 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.398.767 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1570 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:50.410.899 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 66, total running count: 1470 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.411.679 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.414.330 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1571 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.426.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 67, total running count: 1471 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.427.233 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.429.795 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1572 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.441.765 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 68, total running count: 1472 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.442.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.444.117 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1573 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:50.456.988 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 69, total running count: 1473 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:50.457.798 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.459.730 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1574 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:50.472.278 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 70, total running count: 1474 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.473.108 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.475.633 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1575 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.487.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 71, total running count: 1475 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:50.488.543 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.491.229 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1576 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.503.065 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 72, total running count: 1476 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.504.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.506.853 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1577 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.518.686 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 73, total running count: 1477 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.519.518 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.521.241 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1578 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.533.985 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 74, total running count: 1478 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.534.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.536.659 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1579 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.553.552 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 75, total running count: 1479 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.553.970 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.556.776 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1580 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.568.503 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 76, total running count: 1480 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.568.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.571.319 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1581 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:50.583.244 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 77, total running count: 1481 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.583.644 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.585.819 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1582 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:50.598.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 78, total running count: 1482 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:50.598.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.600.351 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1583 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:50.613.239 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 79, total running count: 1483 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.613.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.615.816 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1584 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.628.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 80, total running count: 1484 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.629.508 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.631.348 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1585 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.643.876 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 81, total running count: 1485 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:50.645.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.646.834 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1586 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.659.612 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 82, total running count: 1486 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.660.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.663.507 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1587 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.675.254 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 83, total running count: 1487 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:50.676.347 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.679.018 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1588 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.690.969 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 84, total running count: 1488 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.692.045 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.694.677 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1589 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.706.440 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 85, total running count: 1489 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:50.707.736 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.710.248 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1590 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.722.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 86, total running count: 1490 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.723.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.724.963 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1591 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:50.737.677 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 87, total running count: 1491 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.738.781 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.740.509 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1592 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:50.753.191 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 88, total running count: 1492 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.754.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.755.959 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1593 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.768.583 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 89, total running count: 1493 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.769.716 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.771.421 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1594 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.784.255 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 90, total running count: 1494 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:50.785.265 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.787.922 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1595 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:50.799.698 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 91, total running count: 1495 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.800.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.803.327 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1596 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.815.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 92, total running count: 1496 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.816.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.818.859 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1597 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.831.254 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 93, total running count: 1497 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:50.832.277 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.834.309 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1598 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.846.733 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 94, total running count: 1498 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:50.847.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.849.790 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1599 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.862.447 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 95, total running count: 1499 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.863.403 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.865.350 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1600 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.878.117 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 96, total running count: 1500 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.879.033 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.881.021 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1601 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.893.506 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 97, total running count: 1501 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.894.602 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.897.065 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1602 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:50.909.132 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 98, total running count: 1502 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:50.909.939 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.912.548 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1603 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.924.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 99, total running count: 1503 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.925.262 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.926.860 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1604 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:50.939.856 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 100, total running count: 1504 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.940.854 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.942.556 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1605 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.955.303 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 101, total running count: 1505 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.956.442 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.958.543 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1606 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:50.970.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 102, total running count: 1506 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:50.971.833 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.974.294 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1607 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:50.986.379 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 103, total running count: 1507 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:50.987.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:50.989.788 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1608 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.001.778 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 104, total running count: 1508 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.002.858 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.004.457 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1609 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.017.111 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 105, total running count: 1509 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.018.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.020.222 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1610 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.032.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 106, total running count: 1510 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.033.985 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.035.971 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1611 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.048.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 107, total running count: 1511 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.049.623 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.051.689 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1612 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.064.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 108, total running count: 1512 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:51.065.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.067.319 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1613 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.079.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 109, total running count: 1513 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.080.951 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.082.914 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1614 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.095.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 110, total running count: 1514 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.096.392 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.098.708 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1615 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.110.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 111, total running count: 1515 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:51.112.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.114.540 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1616 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.126.776 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 112, total running count: 1516 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.127.599 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.130.199 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1617 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.142.142 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 113, total running count: 1517 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.143.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.145.776 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1618 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.157.460 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 114, total running count: 1518 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.158.636 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.161.280 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1619 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.173.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 115, total running count: 1519 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.174.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.175.757 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1620 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:51.188.575 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 116, total running count: 1520 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.189.721 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.191.522 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1621 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.204.182 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 117, total running count: 1521 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.205.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.207.306 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1622 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.219.442 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 118, total running count: 1522 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.220.612 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.222.907 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1623 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.235.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 119, total running count: 1523 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.235.900 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.238.529 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1624 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.250.403 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 120, total running count: 1524 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.251.735 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.254.582 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1625 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.266.481 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 121, total running count: 1525 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.267.554 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.270.293 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1626 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.281.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 122, total running count: 1526 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:51.282.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.284.656 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1627 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.297.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 123, total running count: 1527 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:51.298.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.301.383 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1628 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:51.313.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 124, total running count: 1528 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.314.148 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.316.036 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1629 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.328.590 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 125, total running count: 1529 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.329.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.332.024 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1630 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.343.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 126, total running count: 1530 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.344.843 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.347.513 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1631 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.359.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 127, total running count: 1531 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.360.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.361.870 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1632 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.374.503 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 128, total running count: 1532 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.375.437 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.377.682 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1633 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:51.390.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 129, total running count: 1533 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.390.914 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.392.566 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1634 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.405.387 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 130, total running count: 1534 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.406.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.408.163 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1635 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.420.841 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 131, total running count: 1535 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:51.421.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.423.679 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1636 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.436.132 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 132, total running count: 1536 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.437.563 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.439.155 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1637 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.452.079 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 133, total running count: 1537 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.453.210 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.455.826 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1638 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.467.582 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 134, total running count: 1538 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.468.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.471.294 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1639 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.483.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 135, total running count: 1539 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.484.433 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.486.805 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1640 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:51.498.899 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 136, total running count: 1540 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.500.204 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.502.238 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1641 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.514.752 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 137, total running count: 1541 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:51.515.626 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.518.133 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1642 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.530.024 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 138, total running count: 1542 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.531.091 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.533.634 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1643 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.545.372 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 139, total running count: 1543 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.546.686 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.549.299 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1644 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.561.080 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 140, total running count: 1544 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.562.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.563.916 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1645 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.576.609 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 141, total running count: 1545 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.577.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.579.355 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1646 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.592.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 142, total running count: 1546 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:51.593.060 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.594.945 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1647 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.607.645 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 143, total running count: 1547 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.608.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.610.569 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1648 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.623.019 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 144, total running count: 1548 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.624.089 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.626.289 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1649 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.638.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 145, total running count: 1549 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:51.639.617 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.642.258 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1650 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.653.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 146, total running count: 1550 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.654.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.656.855 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1651 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.669.545 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 147, total running count: 1551 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.670.481 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.672.546 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1652 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.684.701 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 148, total running count: 1552 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.685.843 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.688.376 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1653 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.700.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 149, total running count: 1553 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.701.276 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.702.856 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1654 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.715.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 150, total running count: 1554 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:51.716.783 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.718.360 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1655 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.731.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 151, total running count: 1555 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.732.403 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.734.043 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1656 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.746.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 152, total running count: 1556 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.747.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.749.913 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1657 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.762.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 153, total running count: 1557 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.763.210 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.765.657 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1658 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.777.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 154, total running count: 1558 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.778.731 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.781.047 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1659 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.793.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 155, total running count: 1559 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:51.794.378 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.796.748 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1660 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:51.808.815 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 156, total running count: 1560 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.809.574 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.812.329 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1661 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.824.011 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 157, total running count: 1561 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.825.056 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.826.636 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1662 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.839.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 158, total running count: 1562 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:51.840.348 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.842.513 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1663 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.854.938 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 159, total running count: 1563 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.856.065 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.858.178 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1664 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.870.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 160, total running count: 1564 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.871.484 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.873.746 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1665 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.885.771 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 161, total running count: 1565 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.886.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.889.521 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1666 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.901.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 162, total running count: 1566 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.902.517 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.904.096 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1667 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:51.916.864 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 163, total running count: 1567 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.917.937 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.919.790 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1668 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:51.932.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 164, total running count: 1568 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.933.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.935.270 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1669 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.947.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 165, total running count: 1569 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.948.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.950.823 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1670 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.963.146 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 166, total running count: 1570 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:51.964.278 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.966.402 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1671 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.978.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 167, total running count: 1571 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.979.617 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.982.135 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1672 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:51.994.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 168, total running count: 1572 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:51.994.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:51.996.662 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1673 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.009.120 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 169, total running count: 1573 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.010.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.012.577 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1674 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.024.541 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 170, total running count: 1574 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.025.455 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.028.035 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1675 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.040.042 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 171, total running count: 1575 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.041.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.043.704 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1676 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:52.055.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 172, total running count: 1576 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.056.606 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.058.592 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1677 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.071.120 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 173, total running count: 1577 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:52.072.141 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.074.110 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1678 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.086.688 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 174, total running count: 1578 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:52.087.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.089.667 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1679 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.102.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 175, total running count: 1579 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:52.103.045 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.105.537 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1680 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:52.117.544 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 176, total running count: 1580 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:52.118.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.120.092 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1681 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.133.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 177, total running count: 1581 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.133.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.135.665 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1682 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:52.147.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 178, total running count: 1582 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.149.153 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.151.147 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1683 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.163.719 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 179, total running count: 1583 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.164.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.166.639 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1684 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.178.904 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 180, total running count: 1584 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:52.180.019 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.182.115 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1685 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:52.194.436 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 181, total running count: 1585 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:52.195.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.197.621 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1686 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.210.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 182, total running count: 1586 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.210.957 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.213.295 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1687 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.225.356 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 183, total running count: 1587 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:52.226.434 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.228.189 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1688 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.240.864 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 184, total running count: 1588 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.241.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.243.841 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1689 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.256.208 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 185, total running count: 1589 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.257.139 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.259.421 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1690 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.271.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 186, total running count: 1590 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.272.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.274.843 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1691 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.286.979 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 187, total running count: 1591 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.288.164 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.290.540 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1692 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:52.302.550 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 188, total running count: 1592 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.303.782 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.305.376 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1693 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:52.318.314 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 189, total running count: 1593 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:52.319.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.321.213 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1694 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.333.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 190, total running count: 1594 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.334.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.337.042 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1695 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.349.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 191, total running count: 1595 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.350.567 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.352.914 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1696 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.364.968 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 192, total running count: 1596 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.366.471 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.368.582 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1697 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.380.795 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 193, total running count: 1597 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.382.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.384.028 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1698 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.396.527 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 194, total running count: 1598 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.397.713 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.399.497 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1699 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.412.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 195, total running count: 1599 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.413.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.415.335 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1700 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.427.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 196, total running count: 1600 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.428.978 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.431.097 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1701 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:52.443.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 197, total running count: 1601 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.444.533 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.446.836 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1702 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.459.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 198, total running count: 1602 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.460.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.462.330 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1703 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.474.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 199, total running count: 1603 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.475.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.477.889 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1704 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:52.490.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 200, total running count: 1604 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.491.184 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.493.599 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1705 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.505.394 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 201, total running count: 1605 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:52.506.687 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.509.367 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1706 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.520.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 202, total running count: 1606 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.522.149 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.523.712 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1707 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.536.594 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 203, total running count: 1607 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.537.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.540.400 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1708 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:52.552.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 204, total running count: 1608 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.553.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.554.881 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1709 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:52.567.690 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 205, total running count: 1609 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.568.656 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.570.353 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1710 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.583.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 206, total running count: 1610 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.584.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.585.801 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1711 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.598.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 207, total running count: 1611 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:52.599.594 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.601.437 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1712 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:52.614.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 208, total running count: 1612 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.615.048 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.617.078 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1713 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:52.629.458 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 209, total running count: 1613 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.630.510 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.632.932 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1714 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.644.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 210, total running count: 1614 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.645.917 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.648.594 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1715 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.660.376 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 211, total running count: 1615 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.661.629 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.664.218 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1716 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.676.002 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 212, total running count: 1616 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.677.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.678.582 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1717 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:52.691.440 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 213, total running count: 1617 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.692.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.695.155 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1718 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.706.907 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 214, total running count: 1618 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:52.707.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.709.597 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1719 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:52.722.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 215, total running count: 1619 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.723.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.725.199 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1720 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.737.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 216, total running count: 1620 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:52.739.006 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.740.673 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1721 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.753.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 217, total running count: 1621 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.754.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.756.215 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1722 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:52.768.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 218, total running count: 1622 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:52.769.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.771.749 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1723 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:52.784.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 219, total running count: 1623 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:52.785.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.787.461 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1724 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:52.799.986 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 220, total running count: 1624 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:52.800.851 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.803.170 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1725 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.815.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 221, total running count: 1625 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.816.481 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.818.973 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1726 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:52.830.920 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 222, total running count: 1626 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.831.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.834.547 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1727 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:52.846.562 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 223, total running count: 1627 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.847.569 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.849.254 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1728 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.861.968 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 224, total running count: 1628 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.863.120 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.865.209 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1729 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.877.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 225, total running count: 1629 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.878.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.880.806 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1730 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.893.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 226, total running count: 1630 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.895.531 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.897.566 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1731 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.910.044 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 227, total running count: 1631 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.911.232 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.913.322 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1732 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.925.665 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 228, total running count: 1632 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.926.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.929.199 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1733 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.940.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 229, total running count: 1633 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:52.942.271 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.944.630 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1734 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:52.956.601 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 230, total running count: 1634 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.957.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.960.186 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1735 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:52.972.302 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 231, total running count: 1635 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:52.973.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.975.702 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1736 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.987.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 232, total running count: 1636 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:52.988.779 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:52.991.256 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1737 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:53.003.262 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 233, total running count: 1637 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.004.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.006.705 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1738 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.018.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 234, total running count: 1638 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.019.697 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.022.337 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1739 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:53.034.048 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 235, total running count: 1639 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.035.254 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.036.907 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1740 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:53.049.570 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 236, total running count: 1640 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.050.671 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.052.578 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1741 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:53.065.027 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 237, total running count: 1641 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.066.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.068.173 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1742 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:53.080.483 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 238, total running count: 1642 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.081.382 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.083.860 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1743 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.095.985 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 239, total running count: 1643 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:53.096.782 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.099.334 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1744 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.111.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 240, total running count: 1644 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.112.243 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.114.798 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1745 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.126.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 241, total running count: 1645 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.127.828 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.129.472 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1746 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.142.239 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 242, total running count: 1646 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:53.143.202 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.145.230 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1747 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.157.736 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 243, total running count: 1647 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.159.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.161.219 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1748 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.173.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 244, total running count: 1648 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:53.174.687 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.176.682 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1749 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.189.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 245, total running count: 1649 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.190.173 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.192.334 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1750 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.204.488 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 246, total running count: 1650 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.205.637 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.208.075 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1751 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.220.179 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 247, total running count: 1651 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.221.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.223.755 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1752 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.235.827 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 248, total running count: 1652 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.236.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.238.548 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1753 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:53.251.164 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 249, total running count: 1653 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.252.335 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.254.273 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1754 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.266.737 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 250, total running count: 1654 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.267.642 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.269.678 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1755 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.282.233 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 251, total running count: 1655 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.283.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.285.388 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1756 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.297.666 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 252, total running count: 1656 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:53.298.771 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.300.885 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1757 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:53.313.139 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 253, total running count: 1657 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.314.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.316.592 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1758 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.328.831 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 254, total running count: 1658 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.329.596 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.332.162 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1759 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.343.867 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 255, total running count: 1659 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:53.345.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.346.886 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1760 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.359.711 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 256, total running count: 1660 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:53.360.620 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.362.745 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1761 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:53.375.032 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 257, total running count: 1661 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.376.336 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.378.257 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1762 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.390.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 258, total running count: 1662 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.391.696 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.393.852 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1763 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.406.268 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 259, total running count: 1663 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:53.407.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.409.769 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1764 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.421.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 260, total running count: 1664 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.423.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.425.623 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1765 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.437.501 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 261, total running count: 1665 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.438.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.440.038 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1766 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.452.880 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 262, total running count: 1666 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:53.454.662 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.456.915 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1767 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.469.097 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 263, total running count: 1667 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.470.413 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.472.437 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1768 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.484.855 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 264, total running count: 1668 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:53.486.043 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.488.301 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1769 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:53.500.668 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 265, total running count: 1669 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:53.501.633 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.504.016 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1770 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.515.976 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 266, total running count: 1670 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.517.290 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.519.481 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1771 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.531.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 267, total running count: 1671 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.532.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.534.912 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1772 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.547.269 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 268, total running count: 1672 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.548.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.550.527 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1773 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.562.734 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 269, total running count: 1673 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.563.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.566.108 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1774 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:53.578.071 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 270, total running count: 1674 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.579.347 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.581.656 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1775 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.593.899 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 271, total running count: 1675 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.595.058 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.597.265 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1776 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:53.609.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 272, total running count: 1676 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:53.610.363 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.612.969 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1777 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:53.624.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 273, total running count: 1677 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:53.625.864 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.628.542 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1778 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:53.640.282 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 274, total running count: 1678 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.641.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.643.207 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1779 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:53.655.817 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 275, total running count: 1679 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.656.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.658.809 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1780 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.671.374 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 276, total running count: 1680 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.672.331 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.674.249 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1781 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.686.781 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 277, total running count: 1681 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.687.876 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.689.985 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1782 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:53.702.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 278, total running count: 1682 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.704.234 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.706.686 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1783 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.718.733 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 279, total running count: 1683 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.721.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.723.316 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1784 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.735.771 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 280, total running count: 1684 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.737.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.738.748 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1785 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.751.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 281, total running count: 1685 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.752.898 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.755.305 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1786 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.767.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 282, total running count: 1686 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.768.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.770.811 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1787 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:53.782.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 283, total running count: 1687 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.784.267 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.786.235 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1788 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:53.798.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 284, total running count: 1688 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.799.951 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.801.990 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1789 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.814.367 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 285, total running count: 1689 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.815.623 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.817.779 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1790 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.829.998 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 286, total running count: 1690 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.831.096 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.833.737 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1791 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:53.845.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 287, total running count: 1691 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.846.799 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.849.293 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1792 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.861.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 288, total running count: 1692 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.862.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.864.800 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1793 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:53.876.554 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 289, total running count: 1693 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.877.755 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.879.361 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1794 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.892.208 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 290, total running count: 1694 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:53.893.047 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.894.759 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1795 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.907.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 291, total running count: 1695 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.908.744 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.911.369 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1796 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.922.998 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 292, total running count: 1696 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.924.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.926.827 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1797 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.938.610 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 293, total running count: 1697 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.939.690 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.941.229 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1798 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.954.170 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 294, total running count: 1698 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:53.955.071 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.957.483 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1799 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:53.969.498 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 295, total running count: 1699 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.970.785 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.973.012 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1800 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:53.985.145 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 296, total running count: 1700 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:53.986.722 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:53.988.637 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1801 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:54.000.993 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 297, total running count: 1701 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:54.002.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.004.122 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1802 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.016.748 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 298, total running count: 1702 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:54.017.930 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.019.602 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1803 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:54.032.596 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 299, total running count: 1703 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.033.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.035.266 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1804 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.047.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 300, total running count: 1704 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:54.048.960 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.050.912 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1805 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.063.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 301, total running count: 1705 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.064.255 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.066.527 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1806 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.078.492 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 302, total running count: 1706 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:54.079.870 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.082.288 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1807 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.094.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 303, total running count: 1707 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.095.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.097.844 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1808 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.109.751 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 304, total running count: 1708 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.110.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.112.514 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1809 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.125.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 305, total running count: 1709 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.126.303 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.128.341 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1810 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.140.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 306, total running count: 1710 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.141.793 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.144.016 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1811 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.156.263 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 307, total running count: 1711 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.157.387 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.159.654 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1812 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.171.760 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 308, total running count: 1712 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.173.002 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.175.653 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1813 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.187.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 309, total running count: 1713 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.188.473 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.190.182 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1814 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:54.202.959 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 310, total running count: 1714 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:54.203.829 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.205.758 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1815 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:54.218.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 311, total running count: 1715 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.219.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.221.577 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1816 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.233.936 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 312, total running count: 1716 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.234.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.237.148 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1817 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:54.249.304 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 313, total running count: 1717 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.250.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.252.908 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1818 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.264.955 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 314, total running count: 1718 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.266.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.268.606 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1819 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.280.489 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 315, total running count: 1719 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.281.647 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.284.123 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1820 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.295.922 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 316, total running count: 1720 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.297.239 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.299.744 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1821 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.311.808 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 317, total running count: 1721 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.313.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.315.359 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1822 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.327.611 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 318, total running count: 1722 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:54.328.726 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.331.297 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1823 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.343.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 319, total running count: 1723 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.344.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.345.791 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1824 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.358.604 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 320, total running count: 1724 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.359.748 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.362.445 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1825 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:54.374.289 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 321, total running count: 1725 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.375.186 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.377.035 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1826 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.389.450 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 322, total running count: 1726 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:54.390.773 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.392.871 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1827 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.405.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 323, total running count: 1727 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:54.406.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.408.328 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1828 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.420.715 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 324, total running count: 1728 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.421.955 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.423.993 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1829 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.436.335 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 325, total running count: 1729 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.437.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.439.520 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1830 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:54.452.016 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 326, total running count: 1730 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.453.319 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.454.932 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1831 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.467.993 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 327, total running count: 1731 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.469.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.471.731 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1832 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.483.777 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 328, total running count: 1732 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.484.865 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.487.397 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1833 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.499.406 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 329, total running count: 1733 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:54.500.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.503.289 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1834 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.515.394 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 330, total running count: 1734 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.516.606 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.519.040 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1835 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.531.168 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 331, total running count: 1735 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.532.513 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.534.999 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1836 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.547.201 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 332, total running count: 1736 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:54.548.167 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.550.885 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1837 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.562.762 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 333, total running count: 1737 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.563.893 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.565.566 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1838 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.578.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 334, total running count: 1738 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:54.579.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.582.193 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1839 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.594.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 335, total running count: 1739 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.595.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.597.699 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1840 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.609.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 336, total running count: 1740 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:54.610.935 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.613.418 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1841 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.625.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 337, total running count: 1741 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.626.823 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.629.082 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1842 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.641.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 338, total running count: 1742 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:54.642.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.644.849 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1843 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.657.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 339, total running count: 1743 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.658.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.660.708 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1844 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.673.121 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 340, total running count: 1744 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.674.026 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.676.699 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1845 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:54.688.764 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 341, total running count: 1745 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:54.689.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.691.424 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1846 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.704.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 342, total running count: 1746 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.705.748 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.707.430 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1847 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.720.684 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 343, total running count: 1747 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.721.598 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.724.392 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1848 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.736.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 344, total running count: 1748 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.737.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.738.934 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1849 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.752.145 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 345, total running count: 1749 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.753.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.755.838 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1850 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:54.768.218 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 346, total running count: 1750 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.769.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.771.402 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1851 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:54.784.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 347, total running count: 1751 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:54.784.961 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.787.033 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1852 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.799.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 348, total running count: 1752 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.800.750 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.802.836 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1853 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.815.495 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 349, total running count: 1753 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.816.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.818.216 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1854 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.831.396 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 350, total running count: 1754 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.833.086 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.835.464 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1855 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.847.805 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 351, total running count: 1755 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.851.166 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.853.497 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1856 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.865.852 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 352, total running count: 1756 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.868.474 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.870.760 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1857 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.883.008 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 353, total running count: 1757 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.884.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.886.597 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1858 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.899.219 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 354, total running count: 1758 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.900.362 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.902.435 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1859 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:54.915.292 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 355, total running count: 1759 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:54.916.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.918.520 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1860 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.931.072 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 356, total running count: 1760 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.932.033 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.934.633 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1861 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:54.946.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 357, total running count: 1761 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.948.138 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.950.222 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1862 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.962.790 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 358, total running count: 1762 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.964.011 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.965.734 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1863 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:54.978.812 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 359, total running count: 1763 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.979.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.982.541 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1864 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.994.683 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 360, total running count: 1764 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:54.995.657 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:54.998.222 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1865 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.010.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 361, total running count: 1765 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.012.632 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.015.266 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1866 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.027.385 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 362, total running count: 1766 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.028.928 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.030.869 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1867 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.043.701 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 363, total running count: 1767 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:55.044.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.047.560 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1868 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.059.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 364, total running count: 1768 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:55.062.099 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.064.543 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1869 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.076.783 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 365, total running count: 1769 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.078.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.080.472 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1870 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:55.092.716 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 366, total running count: 1770 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.093.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.096.076 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1871 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:55.108.523 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 367, total running count: 1771 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.109.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.111.437 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1872 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.124.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 368, total running count: 1772 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.125.200 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.127.228 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1873 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.140.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 369, total running count: 1773 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.140.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.142.878 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1874 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.155.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 370, total running count: 1774 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.156.549 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.158.464 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1875 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.171.292 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 371, total running count: 1775 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.172.465 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.175.134 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1876 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.187.096 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 372, total running count: 1776 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.190.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.192.917 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1877 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.205.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 373, total running count: 1777 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.206.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.208.639 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1878 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.221.141 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 374, total running count: 1778 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:55.222.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.224.260 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1879 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.236.818 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 375, total running count: 1779 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.237.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.239.903 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1880 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.252.379 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 376, total running count: 1780 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.253.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.255.620 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1881 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:55.267.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 377, total running count: 1781 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.268.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.271.402 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1882 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.283.419 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 378, total running count: 1782 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.284.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.286.819 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1883 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.298.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 379, total running count: 1783 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:55.300.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.302.308 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1884 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.314.748 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 380, total running count: 1784 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.315.602 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.318.063 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1885 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.330.130 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 381, total running count: 1785 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.331.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.333.827 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1886 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.345.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 382, total running count: 1786 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.347.421 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.349.677 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1887 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:55.362.183 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 383, total running count: 1787 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.363.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.366.547 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1888 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.378.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 384, total running count: 1788 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:55.379.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.382.249 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1889 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:55.394.617 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 385, total running count: 1789 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:55.395.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.398.133 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1890 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.410.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 386, total running count: 1790 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.412.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.415.184 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1891 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.427.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 387, total running count: 1791 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:55.429.111 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.431.143 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1892 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:55.443.704 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 388, total running count: 1792 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.444.903 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.446.789 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1893 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.459.574 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 389, total running count: 1793 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.460.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.463.511 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1894 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.475.602 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 390, total running count: 1794 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:55.476.677 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.479.081 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1895 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.491.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 391, total running count: 1795 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.492.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.494.921 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1896 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.507.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 392, total running count: 1796 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:55.508.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.510.813 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1897 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.522.664 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 393, total running count: 1797 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.524.675 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.526.349 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1898 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.539.247 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 394, total running count: 1798 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.541.560 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.544.067 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1899 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.556.043 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 395, total running count: 1799 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.557.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.559.520 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1900 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.572.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 396, total running count: 1800 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.573.608 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.576.317 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1901 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.588.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 397, total running count: 1801 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:55.589.623 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.591.908 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1902 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.604.358 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 398, total running count: 1802 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:55.605.767 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.607.836 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1903 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.620.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 399, total running count: 1803 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.621.609 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.623.458 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1904 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.636.011 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 400, total running count: 1804 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.637.369 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.638.914 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1905 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.651.988 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 401, total running count: 1805 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.654.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.656.814 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1906 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:55.668.837 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 402, total running count: 1806 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:55.671.580 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.673.674 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1907 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.686.193 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 403, total running count: 1807 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:55.687.780 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.689.808 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1908 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:55.702.516 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 404, total running count: 1808 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:55.703.712 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.705.467 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1909 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.718.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 405, total running count: 1809 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:55.719.768 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.722.180 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1910 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.734.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 406, total running count: 1810 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:55.735.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.737.791 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1911 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.750.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 407, total running count: 1811 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.751.850 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.754.475 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1912 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.766.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 408, total running count: 1812 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.768.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.770.447 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1913 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.782.926 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 409, total running count: 1813 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.783.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.786.296 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1914 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.798.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 410, total running count: 1814 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.799.701 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.801.902 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1915 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:55.814.325 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 411, total running count: 1815 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.815.678 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.817.996 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1916 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.830.419 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 412, total running count: 1816 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:55.831.476 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.833.836 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1917 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.846.185 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 413, total running count: 1817 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.847.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.849.408 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1918 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.861.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 414, total running count: 1818 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.862.980 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.865.243 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1919 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.877.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 415, total running count: 1819 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.878.839 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.880.772 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1920 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.893.516 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 416, total running count: 1820 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:55.895.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.897.670 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1921 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.909.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 417, total running count: 1821 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:55.910.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.912.090 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1922 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.925.163 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 418, total running count: 1822 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.926.410 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.928.992 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1923 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.940.972 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 419, total running count: 1823 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.942.392 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.944.900 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1924 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:55.956.993 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 420, total running count: 1824 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:55.958.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.960.633 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1925 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:55.972.790 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 421, total running count: 1825 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:55.973.992 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.976.247 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1926 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:55.988.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 422, total running count: 1826 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:55.989.922 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:55.991.699 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1927 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.004.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 423, total running count: 1827 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.005.550 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.007.453 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1928 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.020.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 424, total running count: 1828 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.021.312 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.023.220 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1929 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.035.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 425, total running count: 1829 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.036.988 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.038.783 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1930 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.051.735 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 426, total running count: 1830 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.053.161 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.055.722 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1931 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.067.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 427, total running count: 1831 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.069.302 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.071.294 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1932 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.083.885 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 428, total running count: 1832 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.085.353 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.086.913 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1933 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:56.100.032 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 429, total running count: 1833 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.101.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.103.723 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1934 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.115.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 430, total running count: 1834 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.116.688 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.118.550 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1935 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.131.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 431, total running count: 1835 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:56.132.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.134.339 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1936 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:56.146.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 432, total running count: 1836 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.147.879 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.150.090 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1937 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.162.539 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 433, total running count: 1837 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.163.443 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.166.078 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1938 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.178.286 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 434, total running count: 1838 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.179.302 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.181.781 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1939 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:56.193.889 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 435, total running count: 1839 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.195.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.197.462 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1940 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.209.732 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 436, total running count: 1840 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.210.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.213.012 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1941 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.225.559 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 437, total running count: 1841 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.226.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.228.694 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1942 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.241.439 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 438, total running count: 1842 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.242.568 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.244.469 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1943 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.257.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 439, total running count: 1843 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.257.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.260.343 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1944 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.272.683 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 440, total running count: 1844 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.273.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.276.062 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1945 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.288.375 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 441, total running count: 1845 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.289.286 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.291.675 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1946 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.303.953 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 442, total running count: 1846 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.304.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.307.356 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1947 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.319.740 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 443, total running count: 1847 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.320.755 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.323.244 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1948 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.335.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 444, total running count: 1848 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.336.562 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.338.797 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1949 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.351.129 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 445, total running count: 1849 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.352.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.354.465 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1950 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:56.367.335 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 446, total running count: 1850 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:56.368.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.370.468 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1951 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.382.996 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 447, total running count: 1851 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.384.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.386.387 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1952 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:56.398.831 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 448, total running count: 1852 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.399.833 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.401.910 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1953 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.414.632 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 449, total running count: 1853 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.415.739 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.417.642 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1954 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.430.478 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 450, total running count: 1854 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.431.419 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.433.661 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1955 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.446.035 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 451, total running count: 1855 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.447.173 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.449.495 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1956 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.461.815 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 452, total running count: 1856 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.462.759 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.464.970 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1957 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.477.287 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 453, total running count: 1857 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.478.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.480.596 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1958 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.492.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 454, total running count: 1858 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.494.016 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.496.105 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1959 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.508.669 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 455, total running count: 1859 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.509.570 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.511.912 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1960 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.524.303 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 456, total running count: 1860 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.525.105 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.526.689 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1961 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:56.539.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 457, total running count: 1861 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.541.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.543.394 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1962 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.555.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 458, total running count: 1862 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.556.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.558.783 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1963 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.571.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 459, total running count: 1863 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.572.340 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.574.262 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1964 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.586.900 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 460, total running count: 1864 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.588.061 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.589.672 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1965 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.602.441 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 461, total running count: 1865 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.603.545 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.605.779 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1966 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.618.337 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 462, total running count: 1866 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.619.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.621.636 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1967 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.633.962 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 463, total running count: 1867 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.634.695 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.637.295 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1968 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.649.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 464, total running count: 1868 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.650.387 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.653.058 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1969 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.665.081 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 465, total running count: 1869 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.665.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.667.905 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1970 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.680.535 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 466, total running count: 1870 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.681.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.683.519 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1971 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.695.809 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 467, total running count: 1871 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.696.816 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.699.151 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1972 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.711.359 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 468, total running count: 1872 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:40:56.712.331 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_1 execution count: 4, execution time: 7323.96 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:40:56.712.458 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_1 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:40:56.712.539 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty epoch: 4 step: 468, loss is 0.44508999586105347 [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:40:56.713.271 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.h:76] ContinueSend] continue send at the beginning of the epoch [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:40:56.714.129 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:40:56.714.187 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:40:56.714.291 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_1 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.714.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.717.087 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1973 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:56.729.361 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 1, total running count: 1873 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.731.579 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.733.785 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1974 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:56.746.338 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 2, total running count: 1874 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.747.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.749.483 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1975 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.762.051 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 3, total running count: 1875 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.763.016 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.765.598 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1976 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.777.632 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 4, total running count: 1876 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:56.778.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.781.148 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1977 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.793.108 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 5, total running count: 1877 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.794.019 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.795.815 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1978 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.808.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 6, total running count: 1878 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.809.807 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.811.673 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1979 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.824.452 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 7, total running count: 1879 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.825.393 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.827.475 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1980 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.839.947 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 8, total running count: 1880 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:56.840.885 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.842.958 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1981 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:56.855.574 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 9, total running count: 1881 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.857.560 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.860.113 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1982 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.872.374 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 10, total running count: 1882 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.873.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.875.722 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1983 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.888.213 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 11, total running count: 1883 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.888.764 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.891.238 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1984 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.903.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 12, total running count: 1884 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.903.811 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.905.650 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1985 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.918.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 13, total running count: 1885 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:56.918.970 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.921.437 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1986 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.933.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 14, total running count: 1886 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.934.503 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.937.186 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1987 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.949.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 15, total running count: 1887 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:56.949.879 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.951.881 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1988 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.964.468 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 16, total running count: 1888 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:56.965.328 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.967.379 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1989 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:56.980.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 17, total running count: 1889 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.980.816 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.983.137 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1990 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:56.995.437 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 18, total running count: 1890 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:56.996.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:56.998.932 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1991 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:57.010.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 19, total running count: 1891 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.012.084 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.013.632 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1992 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.026.634 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 20, total running count: 1892 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.027.469 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.029.306 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1993 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:57.041.909 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 21, total running count: 1893 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.042.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.044.650 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1994 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.057.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 22, total running count: 1894 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.058.440 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.060.339 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1995 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.073.938 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 23, total running count: 1895 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.074.383 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.076.367 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1996 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:57.090.779 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 24, total running count: 1896 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.091.141 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.093.256 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1997 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:57.106.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 25, total running count: 1897 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.106.403 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.108.670 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1998 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.121.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 26, total running count: 1898 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.121.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.124.370 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1999 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.136.717 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 27, total running count: 1899 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:57.137.068 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.138.951 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2000 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.151.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 28, total running count: 1900 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.152.624 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.154.925 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2001 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.167.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 29, total running count: 1901 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:57.168.219 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.170.446 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2002 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.183.020 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 30, total running count: 1902 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.183.507 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.185.261 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2003 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.198.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 31, total running count: 1903 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.199.013 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.201.135 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2004 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.213.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 32, total running count: 1904 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.214.512 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.216.638 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2005 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.229.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 33, total running count: 1905 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:57.229.868 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.232.126 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2006 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.244.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 34, total running count: 1906 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.245.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.247.994 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2007 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.260.424 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 35, total running count: 1907 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.261.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.263.880 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2008 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.276.094 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 36, total running count: 1908 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:57.277.127 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.279.512 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2009 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.291.480 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 37, total running count: 1909 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.292.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.294.983 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2010 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.307.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 38, total running count: 1910 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.308.654 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.310.456 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2011 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:57.323.215 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 39, total running count: 1911 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.324.324 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.326.065 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2012 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.339.057 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 40, total running count: 1912 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.339.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.341.887 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2013 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:57.354.468 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 41, total running count: 1913 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:57.355.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.357.774 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2014 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:57.370.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 42, total running count: 1914 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.371.927 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.374.523 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2015 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:57.386.484 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 43, total running count: 1915 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.387.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.389.194 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2016 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.402.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 44, total running count: 1916 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:57.403.116 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.405.120 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2017 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:57.417.545 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 45, total running count: 1917 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.418.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.420.697 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2018 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:57.433.090 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 46, total running count: 1918 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.433.987 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.436.468 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2019 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.448.480 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 47, total running count: 1919 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.449.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.451.237 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2020 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.464.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 48, total running count: 1920 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:57.465.043 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.466.744 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2021 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.479.843 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 49, total running count: 1921 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:57.480.734 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.483.407 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2022 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:57.495.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 50, total running count: 1922 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.496.693 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.498.939 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2023 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.511.375 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 51, total running count: 1923 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:57.512.206 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.514.588 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2024 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.526.776 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 52, total running count: 1924 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:57.527.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.530.590 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2025 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.542.481 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 53, total running count: 1925 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.543.319 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.544.967 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2026 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.558.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 54, total running count: 1926 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.558.849 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.560.448 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2027 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:57.573.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 55, total running count: 1927 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:57.574.403 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.576.908 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2028 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.588.978 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 56, total running count: 1928 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.590.143 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.592.451 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2029 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.604.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 57, total running count: 1929 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:57.605.746 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.607.818 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2030 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.620.408 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 58, total running count: 1930 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.621.324 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.623.267 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2031 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:57.635.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 59, total running count: 1931 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:57.637.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.639.154 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2032 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.651.576 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 60, total running count: 1932 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.652.693 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.655.068 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2033 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.667.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 61, total running count: 1933 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:57.668.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.670.713 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2034 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:57.682.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 62, total running count: 1934 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.683.730 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.685.380 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2035 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.698.398 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 63, total running count: 1935 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.699.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.701.220 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2036 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.713.977 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 64, total running count: 1936 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.714.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.716.809 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2037 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.729.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 65, total running count: 1937 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.730.235 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.732.358 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2038 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:57.744.843 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 66, total running count: 1938 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.746.381 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.748.085 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2039 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.760.961 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 67, total running count: 1939 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.762.140 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.764.041 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2040 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.776.651 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 68, total running count: 1940 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.777.572 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.779.551 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2041 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.792.199 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 69, total running count: 1941 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.793.030 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.795.203 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2042 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:57.807.768 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 70, total running count: 1942 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.808.737 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.810.788 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2043 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.823.195 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 71, total running count: 1943 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.824.357 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.826.809 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2044 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.838.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 72, total running count: 1944 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.839.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.842.499 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2045 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.854.697 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 73, total running count: 1945 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.855.264 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.857.378 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2046 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.870.030 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 74, total running count: 1946 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.870.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.873.255 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2047 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.885.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 75, total running count: 1947 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.888.184 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.890.107 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2048 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.902.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 76, total running count: 1948 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.903.132 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.905.895 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2049 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:57.917.715 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 77, total running count: 1949 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.918.216 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.920.555 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2050 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.932.865 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 78, total running count: 1950 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.934.168 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.936.101 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2051 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:57.948.791 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 79, total running count: 1951 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.949.932 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.951.669 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2052 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.964.497 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 80, total running count: 1952 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:57.965.283 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.967.686 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2053 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:57.979.948 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 81, total running count: 1953 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:57.980.727 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.983.339 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2054 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:57.995.385 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 82, total running count: 1954 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:57.996.395 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:57.998.855 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2055 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.011.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 83, total running count: 1955 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.011.961 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.014.325 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2056 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:58.026.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 84, total running count: 1956 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:58.027.464 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.029.133 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2057 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:58.042.089 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 85, total running count: 1957 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.042.851 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.045.038 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2058 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.057.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 86, total running count: 1958 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:58.058.517 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.060.575 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2059 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.073.154 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 87, total running count: 1959 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:58.074.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.076.079 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2060 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:58.088.651 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 88, total running count: 1960 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.089.560 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.091.823 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2061 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.104.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 89, total running count: 1961 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:58.105.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.106.643 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2062 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.119.801 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 90, total running count: 1962 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:58.120.574 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.122.302 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2063 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.135.240 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 91, total running count: 1963 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:58.136.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.137.811 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2064 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.150.597 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 92, total running count: 1964 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.151.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.153.566 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2065 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.165.738 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 93, total running count: 1965 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.166.871 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.169.244 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2066 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:58.181.517 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 94, total running count: 1966 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.182.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.184.979 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2067 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.196.987 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 95, total running count: 1967 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:58.198.096 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.200.438 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2068 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:58.212.614 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 96, total running count: 1968 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.213.414 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.215.845 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2069 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.227.722 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 97, total running count: 1969 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:58.228.895 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.231.317 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2070 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.243.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 98, total running count: 1970 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.244.500 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.246.973 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2071 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.259.142 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 99, total running count: 1971 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.259.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.262.552 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2072 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.274.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 100, total running count: 1972 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:58.275.428 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.277.177 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2073 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.289.940 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 101, total running count: 1973 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.290.922 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.292.987 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2074 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.305.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 102, total running count: 1974 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.306.845 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.308.893 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2075 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:58.321.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 103, total running count: 1975 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.322.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.324.922 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2076 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.336.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 104, total running count: 1976 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.337.908 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.340.470 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2077 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.352.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 105, total running count: 1977 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:58.353.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.354.872 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2078 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:58.367.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 106, total running count: 1978 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.368.881 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.370.688 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2079 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:58.383.466 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 107, total running count: 1979 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:58.384.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.386.304 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2080 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.398.942 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 108, total running count: 1980 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.399.884 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.401.816 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2081 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.414.495 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 109, total running count: 1981 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.415.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.417.604 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2082 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:58.430.022 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 110, total running count: 1982 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.431.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.433.052 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2083 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.445.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 111, total running count: 1983 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.446.384 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.448.686 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2084 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.461.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 112, total running count: 1984 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:58.462.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.464.535 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2085 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.476.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 113, total running count: 1985 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.477.444 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.479.180 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2086 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.492.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 114, total running count: 1986 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.493.192 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.494.866 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2087 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.507.805 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 115, total running count: 1987 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.508.617 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.510.779 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2088 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.523.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 116, total running count: 1988 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.523.879 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.526.504 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2089 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:58.538.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 117, total running count: 1989 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:58.539.295 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.540.959 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2090 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.553.994 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 118, total running count: 1990 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.554.863 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.556.911 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2091 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:58.569.371 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 119, total running count: 1991 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:58.570.314 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.572.650 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2092 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:58.584.850 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 120, total running count: 1992 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.585.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.587.302 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2093 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.599.937 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 121, total running count: 1993 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:58.601.139 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.603.267 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2094 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:58.615.851 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 122, total running count: 1994 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.616.856 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.618.799 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2095 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:58.631.491 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 123, total running count: 1995 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.632.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.634.548 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2096 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:58.646.929 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 124, total running count: 1996 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.647.991 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.650.462 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2097 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.662.691 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 125, total running count: 1997 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:58.663.476 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.665.238 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2098 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.678.044 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 126, total running count: 1998 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.679.298 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.681.968 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2099 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:58.693.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 127, total running count: 1999 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.694.834 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.696.403 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2100 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:58.709.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 128, total running count: 2000 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.710.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.712.332 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2101 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.724.817 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 129, total running count: 2001 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.725.830 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.727.889 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2102 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.740.498 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 130, total running count: 2002 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.741.519 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.743.393 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2103 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:58.756.124 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 131, total running count: 2003 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:58.756.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.759.240 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2104 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:58.771.294 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 132, total running count: 2004 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:58.772.202 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.773.823 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2105 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.787.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 133, total running count: 2005 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.787.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.789.468 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2106 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.802.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 134, total running count: 2006 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:58.803.305 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.804.873 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2107 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.818.070 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 135, total running count: 2007 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.818.708 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.820.680 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2108 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:58.833.386 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 136, total running count: 2008 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:58.833.934 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.836.468 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2109 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.848.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 137, total running count: 2009 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:58.849.250 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.851.174 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2110 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.863.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 138, total running count: 2010 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.864.768 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.866.721 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2111 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.879.398 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 139, total running count: 2011 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:58.880.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.882.260 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2112 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.894.548 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 140, total running count: 2012 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.895.543 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.897.893 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2113 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:58.910.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 141, total running count: 2013 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.911.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.913.639 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2114 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:58.926.178 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 142, total running count: 2014 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.926.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.929.550 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2115 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.941.391 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 143, total running count: 2015 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.942.365 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.945.079 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2116 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:58.956.819 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 144, total running count: 2016 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.957.836 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.959.490 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2117 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:58.972.568 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 145, total running count: 2017 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.973.217 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.974.956 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2118 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:58.987.943 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 146, total running count: 2018 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:58.988.861 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:58.991.761 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2119 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.003.354 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 147, total running count: 2019 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.004.300 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.006.443 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2120 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.018.943 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 148, total running count: 2020 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.019.691 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.022.413 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2121 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.034.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 149, total running count: 2021 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:59.035.156 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.037.284 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2122 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.049.956 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 150, total running count: 2022 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.050.762 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.053.010 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2123 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.065.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 151, total running count: 2023 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:59.066.350 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.068.566 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2124 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.080.918 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 152, total running count: 2024 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.082.066 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.084.151 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2125 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.096.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 153, total running count: 2025 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.097.602 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.099.875 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2126 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:59.112.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 154, total running count: 2026 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.112.924 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.114.642 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2127 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.127.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 155, total running count: 2027 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.128.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.130.150 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2128 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.142.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 156, total running count: 2028 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:59.143.742 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.145.521 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2129 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.158.459 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 157, total running count: 2029 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.159.203 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.161.187 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2130 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.173.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 158, total running count: 2030 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.174.822 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.177.014 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2131 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.189.430 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 159, total running count: 2031 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:59.190.255 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.192.880 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2132 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.204.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 160, total running count: 2032 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.205.765 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.208.336 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2133 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.220.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 161, total running count: 2033 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:59.221.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.223.920 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2134 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.235.892 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 162, total running count: 2034 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.236.827 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.239.341 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2135 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.251.274 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 163, total running count: 2035 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:59.252.343 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.254.739 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2136 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:59.267.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 164, total running count: 2036 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.267.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.270.301 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2137 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.282.476 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 165, total running count: 2037 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.283.183 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.285.741 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2138 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.297.889 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 166, total running count: 2038 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:59.298.877 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.301.163 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2139 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:59.313.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 167, total running count: 2039 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.314.427 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.316.580 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2140 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.329.085 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 168, total running count: 2040 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:59.329.863 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.332.002 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2141 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.344.541 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 169, total running count: 2041 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:59.345.442 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.347.662 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2142 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.360.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 170, total running count: 2042 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.360.841 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.363.077 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2143 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.375.285 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 171, total running count: 2043 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.376.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.378.778 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2144 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:59.391.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 172, total running count: 2044 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.391.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.394.533 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2145 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:59.406.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 173, total running count: 2045 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:59.407.350 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.409.031 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2146 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:59.422.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 174, total running count: 2046 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.422.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.424.776 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2147 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.437.505 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 175, total running count: 2047 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.438.373 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.440.646 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2148 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.453.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 176, total running count: 2048 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.453.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.456.396 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2149 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.468.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 177, total running count: 2049 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:59.469.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.471.172 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2150 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.483.865 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 178, total running count: 2050 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.484.820 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.486.803 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2151 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.499.479 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 179, total running count: 2051 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.500.162 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.502.310 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2152 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:59.514.724 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 180, total running count: 2052 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:59.515.814 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.517.923 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2153 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:59.530.536 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 181, total running count: 2053 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:59.531.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.533.464 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2154 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.546.098 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 182, total running count: 2054 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.547.209 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.549.377 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2155 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:59.561.585 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 183, total running count: 2055 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.563.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.565.046 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2156 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.577.828 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 184, total running count: 2056 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.578.854 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.580.493 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2157 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.593.439 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 185, total running count: 2057 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:59.594.676 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.597.192 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2158 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.609.403 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 186, total running count: 2058 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.610.217 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.612.602 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2159 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.624.898 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 187, total running count: 2059 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.625.721 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.628.091 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2160 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.640.101 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 188, total running count: 2060 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:59.641.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.643.539 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2161 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:59.656.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 189, total running count: 2061 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.657.004 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.659.315 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2162 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.671.589 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 190, total running count: 2062 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:59.672.772 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.675.213 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2163 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:59.687.368 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 191, total running count: 2063 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.688.345 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.690.765 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2164 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.702.936 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 192, total running count: 2064 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.703.916 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.706.179 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2165 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.718.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 193, total running count: 2065 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:59.719.680 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.721.769 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2166 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.734.290 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 194, total running count: 2066 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.735.230 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.737.335 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2167 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.749.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 195, total running count: 2067 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.750.691 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.753.297 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2168 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.765.247 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 196, total running count: 2068 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.766.183 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.767.727 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2169 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.780.798 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 197, total running count: 2069 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:59.781.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.783.288 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2170 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.796.173 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 198, total running count: 2070 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.797.186 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.799.007 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2171 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.811.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 199, total running count: 2071 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.812.668 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.814.988 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2172 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:59.827.378 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 200, total running count: 2072 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.828.158 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.830.804 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2173 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.842.985 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 201, total running count: 2073 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.843.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.845.266 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2174 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.858.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 202, total running count: 2074 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.859.315 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.861.287 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2175 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:59.873.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 203, total running count: 2075 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:59.874.695 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.877.012 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2176 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:59.889.510 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 204, total running count: 2076 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.890.300 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.892.456 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2177 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.904.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 205, total running count: 2077 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.905.666 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.907.909 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2178 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.920.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 206, total running count: 2078 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.921.142 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.923.470 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2179 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.935.648 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 207, total running count: 2079 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:40:59.936.625 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.938.851 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2180 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:40:59.951.301 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 208, total running count: 2080 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.952.067 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.954.245 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2181 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.966.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 209, total running count: 2081 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:40:59.967.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.970.010 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2182 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.982.112 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 210, total running count: 2082 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.983.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:40:59.985.835 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2183 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:40:59.997.778 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 211, total running count: 2083 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:40:59.998.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.000.602 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2184 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.013.448 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 212, total running count: 2084 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:00.014.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.016.171 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2185 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:00.029.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 213, total running count: 2085 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:00.030.025 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.031.897 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2186 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.044.660 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 214, total running count: 2086 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.045.796 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.047.660 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2187 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.060.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 215, total running count: 2087 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.061.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.063.475 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2188 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.076.345 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 216, total running count: 2088 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:00.077.353 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.080.004 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2189 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.092.284 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 217, total running count: 2089 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:00.093.075 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.095.668 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2190 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.107.663 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 218, total running count: 2090 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.108.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.110.508 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2191 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:00.123.197 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 219, total running count: 2091 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.124.325 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.126.437 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2192 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.139.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 220, total running count: 2092 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.139.805 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.142.060 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2193 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:00.154.486 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 221, total running count: 2093 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.155.871 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.157.752 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2194 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.170.388 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 222, total running count: 2094 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.171.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.173.562 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2195 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.185.966 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 223, total running count: 2095 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:00.187.122 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.189.162 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2196 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.201.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 224, total running count: 2096 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.202.757 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.204.638 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2197 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.217.523 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 225, total running count: 2097 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.218.396 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.220.114 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2198 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.233.072 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 226, total running count: 2098 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.233.792 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.235.780 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2199 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.248.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 227, total running count: 2099 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.249.419 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.251.740 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2200 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:00.264.127 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 228, total running count: 2100 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:00.265.260 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.267.427 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2201 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.279.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 229, total running count: 2101 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.281.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.282.876 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2202 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.295.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 230, total running count: 2102 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.296.653 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.298.453 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2203 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.311.212 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 231, total running count: 2103 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.312.355 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.314.245 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2204 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.326.938 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 232, total running count: 2104 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.328.008 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.330.237 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2205 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.342.805 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 233, total running count: 2105 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.343.562 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.345.871 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2206 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.358.208 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 234, total running count: 2106 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.359.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.361.670 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2207 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.374.021 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 235, total running count: 2107 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.375.054 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.377.441 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2208 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:00.389.557 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 236, total running count: 2108 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:00.390.847 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.393.307 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2209 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.405.496 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 237, total running count: 2109 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:00.406.477 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.408.992 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2210 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.421.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 238, total running count: 2110 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.422.124 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.424.804 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2211 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.436.530 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 239, total running count: 2111 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.437.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.439.450 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2212 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.452.238 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 240, total running count: 2112 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.453.187 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.454.963 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2213 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:00.467.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 241, total running count: 2113 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:00.468.711 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.470.497 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2214 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.483.277 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 242, total running count: 2114 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.484.263 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.486.330 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2215 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.498.906 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 243, total running count: 2115 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:00.499.796 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.502.295 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2216 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.514.331 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 244, total running count: 2116 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.515.401 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.517.793 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2217 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:00.530.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 245, total running count: 2117 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.530.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.533.633 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2218 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:00.545.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 246, total running count: 2118 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.546.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.548.101 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2219 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.560.788 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 247, total running count: 2119 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.562.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.563.723 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2220 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:00.576.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 248, total running count: 2120 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:00.577.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.579.279 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2221 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:00.592.313 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 249, total running count: 2121 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.593.065 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.594.880 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2222 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:00.607.545 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 250, total running count: 2122 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.608.471 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.610.763 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2223 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:00.623.000 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 251, total running count: 2123 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:00.623.862 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.625.478 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2224 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.638.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 252, total running count: 2124 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.639.221 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.640.943 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2225 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.654.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 253, total running count: 2125 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:00.654.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.656.540 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2226 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.669.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 254, total running count: 2126 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.670.270 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.672.499 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2227 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.684.685 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 255, total running count: 2127 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.685.820 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.688.471 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2228 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:00.700.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 256, total running count: 2128 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:00.701.459 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.703.961 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2229 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.716.154 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 257, total running count: 2129 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.717.203 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.719.447 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2230 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.731.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 258, total running count: 2130 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:00.733.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.735.141 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2231 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.747.779 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 259, total running count: 2131 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.748.741 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.750.814 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2232 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:00.763.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 260, total running count: 2132 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:00.764.184 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.766.795 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2233 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.778.869 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 261, total running count: 2133 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.780.048 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.782.321 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2234 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.794.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 262, total running count: 2134 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.795.577 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.797.792 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2235 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.810.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 263, total running count: 2135 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:00.811.259 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.813.740 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2236 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.825.952 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 264, total running count: 2136 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.826.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.828.413 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2237 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.841.444 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 265, total running count: 2137 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.842.524 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.845.256 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2238 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:00.857.076 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 266, total running count: 2138 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.858.023 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.859.721 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2239 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.872.776 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 267, total running count: 2139 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.873.538 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.875.373 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2240 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.888.074 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 268, total running count: 2140 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:00.889.120 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.891.249 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2241 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.903.724 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 269, total running count: 2141 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.904.958 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.906.759 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2242 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.919.525 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 270, total running count: 2142 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.920.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.922.364 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2243 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.935.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 271, total running count: 2143 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.936.108 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.938.072 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2244 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.950.644 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 272, total running count: 2144 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:00.951.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.953.775 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2245 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:00.966.272 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 273, total running count: 2145 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.967.163 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.969.269 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2246 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.981.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 274, total running count: 2146 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:00.982.742 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:00.984.982 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2247 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:00.997.264 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 275, total running count: 2147 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:00.998.483 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.000.964 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2248 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.013.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 276, total running count: 2148 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.014.157 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.016.425 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2249 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.028.707 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 277, total running count: 2149 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.029.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.031.923 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2250 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.044.263 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 278, total running count: 2150 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.045.066 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.047.342 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2251 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.059.646 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 279, total running count: 2151 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.060.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.062.939 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2252 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:01.075.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 280, total running count: 2152 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.076.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.078.482 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2253 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.091.012 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 281, total running count: 2153 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.091.743 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.094.467 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2254 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.106.287 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 282, total running count: 2154 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:01.107.205 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.109.171 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2255 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.121.817 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 283, total running count: 2155 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.122.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.124.941 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2256 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.137.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 284, total running count: 2156 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.138.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.140.592 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2257 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.152.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 285, total running count: 2157 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.154.088 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.156.420 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2258 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.168.582 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 286, total running count: 2158 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.169.572 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.171.974 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2259 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.184.054 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 287, total running count: 2159 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:01.185.181 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.187.716 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2260 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.199.845 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 288, total running count: 2160 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.200.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.203.356 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2261 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.215.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 289, total running count: 2161 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.216.237 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.217.903 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2262 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.230.780 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 290, total running count: 2162 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.231.715 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.233.558 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2263 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.246.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 291, total running count: 2163 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.247.245 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.248.934 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2264 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:01.261.617 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 292, total running count: 2164 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.262.683 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.264.615 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2265 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.277.292 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 293, total running count: 2165 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.278.254 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.280.514 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2266 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.292.693 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 294, total running count: 2166 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.293.790 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.296.293 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2267 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.308.394 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 295, total running count: 2167 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:01.309.287 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.311.944 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2268 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.323.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 296, total running count: 2168 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.324.782 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.326.510 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2269 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.339.535 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 297, total running count: 2169 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.340.380 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.342.625 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2270 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.355.038 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 298, total running count: 2170 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.355.782 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.358.255 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2271 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.370.253 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 299, total running count: 2171 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:01.371.347 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.373.640 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2272 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.385.922 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 300, total running count: 2172 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.386.806 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.389.328 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2273 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.401.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 301, total running count: 2173 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.402.377 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.404.070 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2274 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.417.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 302, total running count: 2174 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.418.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.419.871 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2275 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.432.784 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 303, total running count: 2175 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.433.825 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.435.603 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2276 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.448.244 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 304, total running count: 2176 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.449.228 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.451.451 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2277 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.463.761 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 305, total running count: 2177 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.464.725 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.467.175 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2278 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.479.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 306, total running count: 2178 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:01.480.030 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.482.683 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2279 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.494.553 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 307, total running count: 2179 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.495.432 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.497.349 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2280 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.510.068 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 308, total running count: 2180 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.510.749 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.513.233 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2281 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.525.311 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 309, total running count: 2181 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.526.371 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.528.944 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2282 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.540.973 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 310, total running count: 2182 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.541.746 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.543.432 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2283 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:01.556.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 311, total running count: 2183 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.557.248 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.559.111 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2284 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.571.815 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 312, total running count: 2184 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.572.554 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.574.727 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2285 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.587.084 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 313, total running count: 2185 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.588.050 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.590.580 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2286 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.602.564 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 314, total running count: 2186 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:01.603.494 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.606.178 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2287 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.618.015 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 315, total running count: 2187 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.618.886 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.620.522 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2288 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.633.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 316, total running count: 2188 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.634.408 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.636.031 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2289 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.649.214 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 317, total running count: 2189 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.649.771 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.651.727 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2290 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.664.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 318, total running count: 2190 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:01.665.137 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.667.681 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2291 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.679.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 319, total running count: 2191 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.680.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.682.165 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2292 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.695.224 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 320, total running count: 2192 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.695.873 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.697.668 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2293 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:01.710.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 321, total running count: 2193 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.711.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.713.447 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2294 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.726.054 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 322, total running count: 2194 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.726.944 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.729.505 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2295 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.741.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 323, total running count: 2195 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.742.334 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.743.958 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2296 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.756.821 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 324, total running count: 2196 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.757.720 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.759.458 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2297 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.772.411 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 325, total running count: 2197 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.773.306 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.775.242 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2298 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:01.787.897 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 326, total running count: 2198 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.788.887 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.791.239 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2299 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.803.555 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 327, total running count: 2199 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.804.432 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.806.817 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2300 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.819.034 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 328, total running count: 2200 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.819.799 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.822.452 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2301 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.834.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 329, total running count: 2201 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.835.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.837.262 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2302 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.850.226 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 330, total running count: 2202 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.850.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.852.947 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2303 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.865.544 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 331, total running count: 2203 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.866.623 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.868.564 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2304 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.881.232 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 332, total running count: 2204 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.882.103 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.884.333 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2305 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.896.759 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 333, total running count: 2205 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.897.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.899.929 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2306 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:01.912.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 334, total running count: 2206 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.913.455 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.915.485 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2307 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.928.202 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 335, total running count: 2207 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.928.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.931.187 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2308 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.943.482 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 336, total running count: 2208 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:01.944.501 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.946.802 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2309 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.959.125 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 337, total running count: 2209 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.960.168 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.962.406 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2310 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.974.571 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 338, total running count: 2210 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:01.975.759 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.978.067 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2311 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:01.990.572 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 339, total running count: 2211 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:01.991.582 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:01.993.622 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2312 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.006.139 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 340, total running count: 2212 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.007.273 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.009.476 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2313 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.021.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 341, total running count: 2213 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.022.832 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.025.173 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2314 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.037.456 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 342, total running count: 2214 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:02.038.158 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.039.739 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2315 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.052.615 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 343, total running count: 2215 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.053.631 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.055.594 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2316 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.068.091 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 344, total running count: 2216 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.069.160 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.071.340 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2317 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.083.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 345, total running count: 2217 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.085.055 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.086.904 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2318 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.099.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 346, total running count: 2218 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.100.603 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.102.659 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2319 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.115.396 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 347, total running count: 2219 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.116.314 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.118.504 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2320 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:02.130.758 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 348, total running count: 2220 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.131.699 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.134.162 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2321 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.146.468 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 349, total running count: 2221 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.147.236 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.149.903 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2322 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.161.835 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 350, total running count: 2222 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.162.913 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.165.620 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2323 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.177.550 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 351, total running count: 2223 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:02.178.544 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.180.392 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2324 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.193.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 352, total running count: 2224 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.194.204 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.195.873 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2325 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:02.208.826 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 353, total running count: 2225 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.209.902 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.211.681 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2326 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.224.461 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 354, total running count: 2226 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.225.426 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.227.684 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2327 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.239.963 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 355, total running count: 2227 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.240.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.243.400 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2328 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.255.390 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 356, total running count: 2228 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.256.279 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.258.855 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2329 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.270.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 357, total running count: 2229 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.271.828 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.274.541 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2330 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.286.460 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 358, total running count: 2230 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.287.679 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.289.291 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2331 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.302.122 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 359, total running count: 2231 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.303.082 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.305.149 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2332 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.317.601 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 360, total running count: 2232 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:02.318.679 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.320.925 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2333 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:02.333.287 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 361, total running count: 2233 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:02.334.331 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.336.911 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2334 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:02.348.816 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 362, total running count: 2234 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.349.644 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.351.494 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2335 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.364.266 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 363, total running count: 2235 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:02.364.938 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.366.999 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2336 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.379.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 364, total running count: 2236 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:02.380.398 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.382.829 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2337 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.395.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 365, total running count: 2237 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.396.044 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.398.583 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2338 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.410.468 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 366, total running count: 2238 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.411.617 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.414.107 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2339 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.426.248 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 367, total running count: 2239 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.427.064 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.429.729 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2340 batch(es) to device, channel name: 95a67c7a-af69-11ee-b370-30fd658829ca [INFO] MD(164046,fffd737fe0f0,python):2024-01-10-11:41:02.429.828 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:596] SendDataToAscend] ExecutionTree finished. Device queue sent number of batches: 2340 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.441.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 368, total running count: 2240 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.442.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:02.457.184 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 369, total running count: 2241 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.457.949 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.472.641 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 370, total running count: 2242 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.473.554 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:02.487.955 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 371, total running count: 2243 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:02.489.483 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.503.950 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 372, total running count: 2244 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.505.029 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.519.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 373, total running count: 2245 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:02.520.559 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.535.238 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 374, total running count: 2246 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.536.096 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:02.550.528 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 375, total running count: 2247 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.551.630 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.566.182 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 376, total running count: 2248 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.567.299 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:02.581.969 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 377, total running count: 2249 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:02.583.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:02.597.711 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 378, total running count: 2250 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.599.078 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.613.673 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 379, total running count: 2251 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.614.730 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.629.320 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 380, total running count: 2252 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:02.630.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.644.941 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 381, total running count: 2253 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.646.114 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:02.660.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 382, total running count: 2254 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.662.033 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.676.753 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 383, total running count: 2255 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:02.677.575 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.692.158 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 384, total running count: 2256 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.693.062 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.707.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 385, total running count: 2257 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.708.706 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.723.260 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 386, total running count: 2258 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.724.226 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.738.763 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 387, total running count: 2259 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.739.756 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.754.473 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 388, total running count: 2260 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:02.755.180 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.769.982 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 389, total running count: 2261 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.770.971 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.785.509 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 390, total running count: 2262 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.786.321 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.800.923 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 391, total running count: 2263 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.801.774 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.816.316 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 392, total running count: 2264 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.817.427 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:02.832.093 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 393, total running count: 2265 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.833.024 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.847.537 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 394, total running count: 2266 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.848.526 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.863.220 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 395, total running count: 2267 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:02.864.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.878.729 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 396, total running count: 2268 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:02.879.739 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.894.487 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 397, total running count: 2269 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.895.122 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.909.490 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 398, total running count: 2270 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:02.910.637 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:02.925.212 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 399, total running count: 2271 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:02.927.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:02.941.820 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 400, total running count: 2272 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:02.944.502 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.959.257 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 401, total running count: 2273 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:02.960.358 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:02.974.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 402, total running count: 2274 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.975.961 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:02.990.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 403, total running count: 2275 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:02.991.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:03.006.210 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 404, total running count: 2276 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.007.591 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.022.310 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 405, total running count: 2277 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:03.023.364 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:03.037.983 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 406, total running count: 2278 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.038.846 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.053.710 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 407, total running count: 2279 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.054.418 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.069.227 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 408, total running count: 2280 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.070.246 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.084.903 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 409, total running count: 2281 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.085.732 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.100.482 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 410, total running count: 2282 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:03.101.499 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.116.163 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 411, total running count: 2283 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.117.066 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.131.636 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 412, total running count: 2284 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.132.559 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.147.258 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 413, total running count: 2285 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.148.159 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.162.628 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 414, total running count: 2286 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.163.894 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.178.565 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 415, total running count: 2287 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.179.223 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:03.193.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 416, total running count: 2288 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.194.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.209.121 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 417, total running count: 2289 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.210.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.224.614 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 418, total running count: 2290 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.225.504 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.240.003 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 419, total running count: 2291 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.240.915 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.255.595 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 420, total running count: 2292 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.256.354 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.271.164 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 421, total running count: 2293 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.272.144 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.286.639 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 422, total running count: 2294 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.287.467 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.301.990 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 423, total running count: 2295 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.302.876 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.317.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 424, total running count: 2296 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.318.203 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.332.857 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 425, total running count: 2297 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.333.463 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.348.109 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 426, total running count: 2298 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.348.844 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.363.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 427, total running count: 2299 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.364.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.378.776 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 428, total running count: 2300 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.379.569 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.393.880 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 429, total running count: 2301 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.394.850 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.409.397 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 430, total running count: 2302 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:03.410.162 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.424.622 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 431, total running count: 2303 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.425.679 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.440.242 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 432, total running count: 2304 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.441.018 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.455.781 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 433, total running count: 2305 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.459.629 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.474.192 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 434, total running count: 2306 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.475.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:03.490.099 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 435, total running count: 2307 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.491.115 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.505.824 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 436, total running count: 2308 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.506.723 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.521.216 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 437, total running count: 2309 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.522.489 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.537.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 438, total running count: 2310 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.537.896 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.552.322 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 439, total running count: 2311 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.553.416 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.568.014 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 440, total running count: 2312 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.569.254 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.583.788 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 441, total running count: 2313 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:03.584.942 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.599.641 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 442, total running count: 2314 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.600.697 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:03.615.264 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 443, total running count: 2315 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.616.031 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.630.735 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 444, total running count: 2316 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.631.432 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.645.829 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 445, total running count: 2317 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.646.769 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.661.366 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 446, total running count: 2318 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.662.171 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.676.621 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 447, total running count: 2319 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.677.667 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.692.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 448, total running count: 2320 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.693.083 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.707.652 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 449, total running count: 2321 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.708.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.723.110 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 450, total running count: 2322 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.723.765 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.738.342 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 451, total running count: 2323 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.739.092 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.758.169 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 452, total running count: 2324 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.758.527 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.773.635 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 453, total running count: 2325 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.773.995 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:03.788.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 454, total running count: 2326 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.789.520 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.804.091 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 455, total running count: 2327 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.806.069 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:03.820.809 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 456, total running count: 2328 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.822.095 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.836.665 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 457, total running count: 2329 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.837.643 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.852.326 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 458, total running count: 2330 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.853.255 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.867.737 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 459, total running count: 2331 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.868.720 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.883.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 460, total running count: 2332 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.884.742 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.899.333 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 461, total running count: 2333 [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.903.680 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:03.918.172 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 462, total running count: 2334 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.920.449 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:03.935.079 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 463, total running count: 2335 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.936.387 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.951.000 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 464, total running count: 2336 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.952.261 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.966.677 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 465, total running count: 2337 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:03.967.912 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.982.422 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 466, total running count: 2338 [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:03.983.722 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:03.998.225 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 467, total running count: 2339 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:04.000.460 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_1_DeviceDSActor_1) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:04.015.188 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_1_LoopCountActor) running, loop count: 468, current count: 468, total running count: 2340 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:04.016.379 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_1 execution count: 5, execution time: 7301.98 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:04.016.540 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_1 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.016.664 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty epoch: 5 step: 468, loss is 0.162714883685112 [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.017.411 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.h:76] ContinueSend] continue send at the beginning of the epoch [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.017.528 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.017.741 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.068.687 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at t10k-images-idx3-ubyte. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.068.760 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at t10k-labels-idx1-ubyte. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.068.910 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.070.977 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.071.109 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.071.133 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.071.152 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.071.191 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.071.268 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.071.298 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.071.313 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.071.335 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.071.354 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.071.412 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.071.427 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.071.453 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.071.491 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.071.880 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at t10k-images-idx3-ubyte. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.071.912 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at t10k-labels-idx1-ubyte. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.232.547 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.232.666 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.244.340 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.244.471 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.244.493 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.244.512 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.244.576 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.244.650 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.244.679 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.244.693 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.244.714 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.244.733 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.244.767 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.244.782 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.244.805 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.244.837 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.245.197 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at t10k-images-idx3-ubyte. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.245.225 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at t10k-labels-idx1-ubyte. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.391.639 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.391.757 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.402.994 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.403.142 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.403.166 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.403.185 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.403.221 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.403.295 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.403.324 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.403.339 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.403.361 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.403.380 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.403.415 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.403.430 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.403.452 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.403.484 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.403.833 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] ME(164046:281472877868096,MainProcess):2024-01-10-11:41:04.404.535 [mindspore/dataset/engine/datasets.py:4269] queue_name is newly generated. value is 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.406.395 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:145] Compile] Input plan: +-Repeat(count:1) | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.406.503 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:160] Compile] Plan before PrePass: +-Top | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.406.525 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:98] PrePass] Prepare PrePass loops. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.406.539 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.406.570 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.406.638 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.406.663 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.406.677 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.406.697 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.406.716 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:124] PrePass] PrePass completed. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.406.748 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:163] Compile] Plan after PrePass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.406.762 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:133] PostPass] Running repeat pass. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.406.783 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:137] PostPass] Repeat pass completed. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.406.824 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter_lite.cc:166] Compile] Plan after PostPass: +-Top | +-Batch(batch_size:32 drop_remainder:true) | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.407.144 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.407.298 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1729] InitExecDatasetVm] Start InitDataSet Entry [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:04.407.457 [mindspore/ccsrc/backend/graph_compiler/transform.cc:573] CreateBackend] CreateBackend is: ge [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:04.407.481 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:04.407.495 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEBUG(164046,ffff82e63440,python):2024-01-10-11:41:04.407.549 [mindspore/ccsrc/debug/debugger/debugger.cc:129] Init] Debugger got device_id: 3 [INFO] DEBUG(164046,ffff82e63440,python):2024-01-10-11:41:04.407.564 [mindspore/ccsrc/debug/debugger/debugger.cc:131] Init] Debugger got device_target: Ascend [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:04.407.614 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:04.407.628 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.407.639 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1252] SetRunMode] Run graph mode with kernel by kernel by configuration. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:04.407.668 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:546] CompileGraphs] Status record: start compile function graph: _anonymous__377 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.407.731 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unify_mindir_pm_0_erase_invalid_micro_depend in 4.97 us [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:04.407.810 [mindspore/ccsrc/utils/anfalgo.cc:1736] IsNodeOutputDynamicShape] Invalid base shape, node: Default/Return-op0_6 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:04.407.861 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:04.407.876 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:04.407.906 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:04.407.918 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:04.407.936 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:04.407.948 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:04.407.988 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:727] CompileGraph] Compile graph: _anonymous__377, Split segments size: 2 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:04.408.017 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:762] CompileGraphFromSegment] Compile normal segment, the first node: @_anonymous__377:CNode_378{[0]: ValueNode InitDataSetQueue} [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:04.408.065 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:421] CompileGraph] Status record: start compile graph. [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:04.408.095 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:1940] ConstructKernelGraph] Create graph: 2 [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:04.408.164 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:2697] ConstructOutput] Output:@_anonymous__377:CNode_378{[0]: ValueNode InitDataSetQueue} [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.408.375 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:193] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:2 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.408.434 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 6.03 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.408.502 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 40.21 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.408.606 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:154] CommonUnifyMindIR] start common unify mindir opt graph:2 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.408.645 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.408.708 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 57.08 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.408.725 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.408.757 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 28.13 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.408.774 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.408.788 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 14.12 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.408.800 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.408.840 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 36.38 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.408.954 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:79] BackendCommonOptimization] Status record: start common optimization. graph id: 2 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.408.995 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: add_dynamic_shape_attr [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.042 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_0_add_dynamic_shape_attr in 42.16 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.071 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.111 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_1_convert_dynamic_broadcast_to in 35.79 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.153 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_2_convert_const_input_to_attr in 22.48 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.188 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_3_custom_op_const_input_to_attr in 16.85 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.220 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 13.86 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.287 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 48.01 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.306 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 0.76 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.348 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_7_flatten_concat_fission in 24.34 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.386 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 19.08 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.422 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_9_broadcast_to_fusion in 17.52 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.457 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 16.55 us [WARNING] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.558 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:92] BackendCommonOptimization] [PROF]backend_common_optimization costs 604 usec. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.573 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:99] BackendCommonOptimization] Status record: end common optimization. graph id: 2 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.775 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_0_renorm_split in 31.61 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.798 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.870 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_1_reduce_axis_update in 65.09 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.889 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim HistogramFixedWidth not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.904 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_2_histogram_fixed_width_fusion in 14.55 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.921 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim ClipByNorm not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.935 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_3_clip_by_norm_fission in 15.78 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.962 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.975 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_4_tensor_arry_add_flow_cond1 in 15.97 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.409.988 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayGather not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.001 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_5_tensor_arry_add_flow_cond2 in 11.28 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.013 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.026 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_6_ge_tensor_arry_cast_index in 11.42 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.039 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArray not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.052 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_7_tensor_array_prepare in 11.85 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.064 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.097 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_8_space_to_batch_nd_attr_update in 29.53 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.114 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.140 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_9_batch_to_space_nd_attr_update in 23.98 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.154 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: avg_pool_grad_unify_mindir [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.193 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_10_avg_pool_grad_unify_mindir in 35.77 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.240 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_11_adam_weight_decay_unify_mindir in 28.14 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.281 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_12_cdist_fission in 21.1 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.323 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_13_cdist_grad_fission in 21.07 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.392 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_14_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 47.72 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.444 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_15_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 31.23 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.495 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_16_sparse_softmax_cross_entropy_with_logits_unify_mindir in 20.61 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.532 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_17_dropout_unify_mindir1 in 18.15 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.549 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.591 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_18_dropoutgrad_unify_mindir in 38.41 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.611 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.626 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_19_neighbor_exchange_unify_mindir in 16.36 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.639 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.652 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_20_neighbor_exchange_v2_unify_mindir in 12.19 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.665 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.680 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_21_neighbor_exchange_v2_grad_unify_mindir in 13.44 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.694 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.708 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_22_all_to_all_unify_mindir in 14.53 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.723 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim QuantDTypeCast not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.737 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_23_quant_dtype_cst_adjust in 14.72 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.752 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim FSEDecode not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.764 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_24_fse_decode_adjust in 13.31 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.778 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.791 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_25_bn_split in 13.08 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.803 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.857 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_26_bn_grad_unify_mindir in 50.22 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.874 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.898 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_27_bn_grad_split in 23.6 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.912 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.924 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_28_batchnorm_to_bninfer in 11.5 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.937 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.949 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_29_batchnormgrad_to_bninfergrad in 11.16 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.962 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.410.974 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_30_batch_norm_grad_infer_fission in 11.31 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.411.018 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_31_ascend_mindir_op_adapter in 27.42 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.411.037 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_32_DropoutGenMask in 1.2 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.411.078 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_33_lamb_fission_ge in 23.79 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.411.132 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_34_print_insert_placeholder_for_tensor_name in 33.96 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.411.173 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_35_getnext_for_ge in 22.4 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.411.212 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_36_sync_bn_split in 20.28 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.411.250 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_37_sync_bn_grad_split in 18.65 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.411.291 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_38_adaptive_max_pool2d_ge_fusion in 21.31 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.411.327 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_39_avg_pool_grad_for_ge in 17.46 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.411.347 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:260] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.411.448 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_40_FlashAttentionFusionV1 in 99.37 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.411.469 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:374] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.411.587 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_41_FlashAttentionFusionV1 in 113.74 us [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:04.411.931 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:04.411.959 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:04.411.972 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:132] GetRunMode] RunMode::kKernelMode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.412.073 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 61.2 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.412.339 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_0_seed_adapter in 52.41 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.412.369 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: env_op_attr_update [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.412.409 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_1_env_op_attr_update in 36.11 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.412.451 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_2_process call inline in 23.16 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.412.482 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_3_expander_fallback in 10.9 us [WARNING] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.412.560 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:139] GEBackendOptimizeACL] [PROF]ascend_backend_optimize_acl costs 291 usec. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:04.412.597 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] InitDataSetQueue is not defined in opdef. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.412.746 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 9.25 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.412.815 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 43.03 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.412.867 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 30.43 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.412.939 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 49.86 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.412.979 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_4_insert_type_transform_op in 17.03 us [WARNING] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.413.061 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:179] GEBackendOptimizeACLAfterKernelSelect] [PROF]ascend_backend_optimize_acl_after_kernel_select costs 364 usec. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.413.118 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_0_DropoutGenMask in 0.67 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.413.159 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_1_cse in 19.51 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.413.232 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_redundant_op in 30.94 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.413.259 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_3_eliminate_maketuple_getitem in 7.76 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.413.276 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_4_MergeTransData in 0.71 us [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:04.413.400 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive InitDataSetQueue [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:04.413.485 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive InitDataSetQueue [WARNING] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:04.413.503 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:262] CreateKernel] [PROF]create_kernel costs 112 usec. [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:04.413.537 [mindspore/ccsrc/backend/common/session/session_basic.cc:1140] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 2 start [INFO] DEBUG(164046,ffff82e63440,python):2024-01-10-11:41:04.413.551 [mindspore/ccsrc/debug/summary/summary.cc:33] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 2 start [INFO] DEBUG(164046,ffff82e63440,python):2024-01-10-11:41:04.413.564 [mindspore/ccsrc/debug/summary/summary.cc:54] RecurseSetSummaryNodesForAllGraphs] The total summary nodes is: 0 for graph: 2 [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:04.413.612 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:891] PrintGraphExecuteOrder] Graph 2 execution order: [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:04.413.649 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[0], node name[Default/InitDataSetQueue-op0_6], logic id[4294967295], stream id[0], node info[@kernel_graph_2:CNode_378{[0]: ValueNode InitDataSetQueue}] [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.413.677 [mindspore/ccsrc/backend/common/somas/somas.cc:143] Assign] Start Somas Assign for graph 2 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.413.712 [mindspore/ccsrc/backend/common/somas/somas.cc:151] Assign] Somas Configure success, configuration info: Device Name: Ascend Run by execution order: 1 Enable debug log: 0 Debug log path: . [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.413.726 [mindspore/ccsrc/backend/common/somas/somas.cc:154] Assign] Start Initialize SOMAS Model [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.413.760 [mindspore/ccsrc/backend/common/somas/somas.cc:1065] GraphOutputProcess] Set 0 graph output tensors' aligned size to 0. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.413.790 [mindspore/ccsrc/backend/common/somas/somas.cc:1135] RefNodeProcess] Special Tensor total size: RefNode: input 0 output 0 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.413.808 [mindspore/ccsrc/backend/common/somas/somas.cc:551] InitSomasModel] No Tensor from graph 2 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.413.819 [mindspore/ccsrc/backend/common/somas/somas.cc:157] Assign] End Initialize SOMAS Model [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:04.413.829 [mindspore/ccsrc/backend/common/somas/somas.cc:160] Assign] No Somas Tensor in graph 2 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:04.413.840 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:310] DoSomas] Somas allocate success for graph 2 somas size: 0 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:04.413.870 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:801] CreateDeviceAddress] Status record: start create device address. graph id: 2 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:04.413.913 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:808] CreateDeviceAddress] Status record: end create device address. graph id: 2 [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:04.413.939 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1129] CacheGraphOutputToFrontNodeWithIndex] Get graph backend output nodes. [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:04.413.958 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1137] CacheGraphOutputToFrontNodeWithIndex] Get graph front output nodes. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:04.413.974 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:507] CompileGraph] Status record: end compile graph. graph id: 2 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:04.414.042 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:805] CompileGraphFromSegment] Compile cut segment, the cut node: @_anonymous__377:CNode_379{[0]: ValueNode Return, [1]: CNode_378} [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:04.414.096 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:521] Transform] Graph(kernel_graph_2) transforms actor begin, strategy:pipeline_with_execution_order [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:04.414.149 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1232] BuildLoopCountActor] Create loop count actor: kernel_graph_2_LoopCountActor [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:04.414.169 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1265] BuildOutputActor] Create output actor: kernel_graph_2_OutputActor [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:04.414.186 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1285] BuildDataPrepareActor] Create data prepare actor: kernel_graph_2_DataPrepareActor [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:04.414.211 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1518] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_2 start. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:04.414.226 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1520] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_2 end. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:04.414.286 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 1_actor_set_kernel_graph_2_memory_actor_insert in 1.6 us [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:04.414.306 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 2_actor_set_kernel_graph_2_invalid_data_arrow_elimination in 1.08 us [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:04.414.329 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 3_actor_set_kernel_graph_2_multi_actor_fusion in 8.56 us [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:04.414.344 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 4_actor_set_kernel_graph_2_batch_data_arrow_fusion in 0.830005 us [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:04.414.359 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:554] Transform] Graph(kernel_graph_2) transforms actor end. [WARNING] VM(164046,ffff82e63440,python):2024-01-10-11:41:04.414.405 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:612] CompileGraphs] [PROF]compile_func_graph costs 6718 usec. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:04.414.425 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:615] CompileGraphs] Status record: end compile function graph: _anonymous__377, produce actor: kernel_graph_2 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:04.414.454 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_2 [INFO] GE(164046,python):2024-01-10-11:41:04.508.702 [graph_var_manager.cc:1424][EVENT]167352 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:41:04.508.805 [graph_manager.cc:1248][EVENT]167352 PreRun:PreRun start: graph node size 1, session id 31, graph id 30, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:41:04.509.455 [atrace_api.c:28](tid:167352) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:41:04.509.514 [trace_rb_log.c:84](tid:167352) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:41:04.509.528 [atrace_api.c:32](tid:167352) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:41:04.509.547 [client_manager.cpp:157][SetProfilingCallback][tid:167352] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:41:04.510.335 [parallel_partitioner.cc:165][EVENT]167352 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.510.375 [parallel_partitioner.cc:178][EVENT]167352 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.510.420 [graph_prepare.cc:1378][EVENT]167352 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.510.566 [graph_manager.cc:1050][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [159] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.510.588 [graph_manager.cc:1052][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.510.650 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [1] [INFO] GE(164046,python):2024-01-10-11:41:04.510.678 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.510.734 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [44] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.510.747 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.510.812 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.510.825 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.510.837 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.510.955 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.510.976 [graph_manager.cc:1054][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [375] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.511.246 [graph_manager.cc:1055][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [257] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.511.787 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:41:04.511.811 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [3] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.511.822 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.511.831 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [107] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.511.840 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [9] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.511.849 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:41:04.511.858 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.511.867 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.511.876 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.048 [graph_manager.cc:1056][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [1755] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.513.109 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondRemovePass is [3] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.126 [graph_prepare.cc:1982][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [43] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.513.280 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:41:04.513.298 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.308 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.318 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferShapePass is [49] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.327 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [5] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.335 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [1] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:41:04.513.344 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [0] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.352 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.361 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of InferValuePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.390 [graph_prepare.cc:1983][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [252] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.513.414 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.513.436 [graph_prepare.cc:1984][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [29] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.513.451 [graph_prepare.cc:1985][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.513.465 [graph_prepare.cc:1986][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.513.475 [graph_prepare.cc:1987][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.513.490 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.513.501 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.513.514 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.513.584 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.598 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.607 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.616 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.625 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.633 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssertPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.645 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.653 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.661 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.670 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.678 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.735 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SnapshotPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.747 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.756 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [4] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.764 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.773 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.513.802 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.513.815 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.513.842 [graph_prepare.cc:1988][EVENT]167352 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [357] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.513.857 [graph_manager.cc:1065][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [774] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.527.371 [graph_manager.cc:1077][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [13496] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.527.420 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.527.445 [graph_manager.cc:1080][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [37] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.114 [graph_manager.cc:1081][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [2655] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.157 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.170 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.181 [graph_manager.cc:1082][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [32] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.210 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.224 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.237 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.273 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [25] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.287 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.299 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.313 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.344 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [21] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.361 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.377 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.405 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.420 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.432 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.441 [graph_manager.cc:2700][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [236] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.518 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.530.530 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.530.539 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.530.548 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.530.557 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.530.565 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CastRemovePass is [7] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.530.574 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.530.582 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.530.591 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.530.600 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.530.608 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [5] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.530.617 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.530.625 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [0] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.530.633 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.530.642 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.530.652 [graph_manager.cc:2741][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [192] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.661 [graph_manager.cc:2752][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.683 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.693 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.715 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.731 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.742 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.754 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.774 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.788 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.800 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.811 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.826 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.837 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.849 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.861 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.870 [graph_manager.cc:2810][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [191] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.892 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of IdentityPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.530.904 [graph_manager.cc:2821][EVENT]167352 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [25] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.530.931 [graph_manager.cc:1087][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [734] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.531.066 [graph_manager.cc:1088][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [122] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.531.100 [graph_manager.cc:1089][EVENT]167352 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.531.116 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.531.129 [graph_manager.cc:1097][EVENT]167352 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:41:04.531.149 [graph_manager.cc:3325][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.531.304 [engine_place.cc:144][EVENT]167352 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.531.328 [engine_place.cc:144][EVENT]167352 Run:The time cost of aicpu_ascend_kernel::CheckSupported is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.531.388 [graph_manager.cc:3351][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [226] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.531.404 [graph_manager.cc:3364][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.531.466 [engine_partitioner.cc:1139][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.531.482 [engine_partitioner.cc:1142][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.531.586 [engine_partitioner.cc:1148][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [94] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.531.611 [engine_partitioner.cc:1155][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.531.648 [engine_partitioner.cc:1164][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.531.681 [graph_manager.cc:3405][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [264] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.531.698 [graph_manager.cc:3412][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.352 [graph_manager.cc:3422][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [1639] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.383 [graph_manager.cc:3428][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.494 [graph_manager.cc:3467][EVENT]167352 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [93] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.512 [graph_manager.cc:3377][EVENT]167352 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [2096] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.527 [graph_manager.cc:1106][EVENT]167352 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [2385] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.540 [graph_manager.cc:1115][EVENT]167352 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:41:04.533.562 [graph_manager.cc:1130][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.593 [graph_manager.cc:1131][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.614 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.630 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.649 [graph_manager.cc:2837][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [41] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.708 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.533.722 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.533.731 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of CondRemovePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.533.740 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of BitcastPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.533.749 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.533.757 [base_pass.cc:339][EVENT]167352 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:04.533.767 [graph_manager.cc:2864][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [102] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.779 [graph_manager.cc:2872][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.796 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.810 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.823 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.837 [compile_nodes_pass.cc:88][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.847 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.858 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.884 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.905 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.917 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.930 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.941 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.950 [graph_manager.cc:2927][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [157] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.966 [graph_manager.cc:2937][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.533.997 [graph_manager.cc:2943][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.534.012 [graph_manager.cc:2950][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.534.433 [graph_manager.cc:2958][EVENT]167352 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [27] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.534.463 [graph_manager.cc:1132][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [859] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.534.575 [graph_manager.cc:1135][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [97] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.534.611 [graph_manager.cc:2975][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.534.676 [graph_manager.cc:2981][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [52] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.534.691 [pass_manager.cc:82][EVENT]167352 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.534.701 [graph_manager.cc:2986][EVENT]167352 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.534.710 [graph_manager.cc:1136][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [118] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.534.784 [graph_manager.cc:3555][EVENT]167352 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [49] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.534.834 [engine_partitioner.cc:1139][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.534.848 [engine_partitioner.cc:1142][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.534.913 [engine_partitioner.cc:1148][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [55] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.534.935 [engine_partitioner.cc:1155][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.534.964 [engine_partitioner.cc:1164][EVENT]167352 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.534.984 [graph_builder.cc:865][EVENT]167352 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [172] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.535.043 [graph_builder.cc:288][EVENT]167352 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [43] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.535.206 [graph_builder.cc:293][EVENT]167352 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [150] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.535.397 [model_builder.cc:1133][EVENT]167352 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [99] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.535.606 [block_mem_assigner.cc:4069][EVENT]172112 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164046,python):2024-01-10-11:41:04.535.606 [block_mem_assigner.cc:4069][EVENT]172111 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164046,python):2024-01-10-11:41:04.535.889 [model_builder.cc:1144][EVENT]167352 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [459] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.535.915 [model_builder.cc:1152][EVENT]167352 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.535.930 [model_builder.cc:1157][EVENT]167352 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.536.037 [model_builder.cc:1167][EVENT]167352 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [96] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.536.055 [model_builder.cc:1174][EVENT]167352 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.536.074 [model_builder.cc:1180][EVENT]167352 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.536.114 [model_builder.cc:1184][EVENT]167352 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [29] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.536.133 [graph_builder.cc:304][EVENT]167352 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [906] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:41:04.536.225 [logger.cc:1071] 167352 ModelBindStream: model_id=576, stream_id=1857, flag=0. [INFO] GE(164046,python):2024-01-10-11:41:04.536.287 [task_generator.cc:804][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.536.336 [task_generator.cc:805][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.536.849 [task_generator.cc:814][EVENT]167352 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [500] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.536.863 [task_generator.cc:954][EVENT]167352 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [583] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.536.920 [task_generator.cc:967][EVENT]167352 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [31] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:41:04.536.939 [logger.cc:1084] 167352 ModelUnbindStream: model_id=576, stream_id=1857, [INFO] GE(164046,python):2024-01-10-11:41:04.536.997 [graph_builder.cc:310][EVENT]167352 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [851] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.537.100 [graph_manager.cc:1152][EVENT]167352 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2373] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.537.116 [graph_manager.cc:1164][EVENT]167352 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:41:04.537.149 [graph_manager.cc:1271][EVENT]167352 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [26895] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.537.160 [graph_manager.cc:1272][EVENT]167352 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:41:04.537.479 [atrace_api.c:93](tid:167352) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:41:04.537.496 [atrace_api.c:95](tid:167352) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:41:04.538.249 [model_introduction.cc:236][EVENT]167352 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(164046,python):2024-01-10-11:41:04.538.272 [model_introduction.cc:294][EVENT]167352 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(164046,python):2024-01-10-11:41:04.538.284 [model_introduction.cc:366][EVENT]167352 Data:model io_info size:0 [INFO] GE(164046,python):2024-01-10-11:41:04.540.366 [graph_converter.cc:838][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [689] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.540.432 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.540.644 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [195] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.540.705 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [42] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.540.718 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [56] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.540.741 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.540.765 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.540.783 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of ZeroCopy is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.540.816 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CEM is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.540.858 [copy_flow_launch_fuse.cc:395][EVENT]167352 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.540.869 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [42] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.540.886 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.540.904 [base_optimizer.cc:70][EVENT]167352 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.540.917 [graph_converter.cc:849][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [512] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.541.033 [graph_converter.cc:853][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [107] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.541.434 [graph_converter.cc:857][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [388] micro second. [INFO] GE(164046,python):2024-01-10-11:41:04.541.502 [graph_converter.cc:862][EVENT]167352 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [49] micro second. [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:04.542.803 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_2_LoopCountActor) running, loop count: 1, current count: 1, total running count: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:04.543.067 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_2 execution count: 1, execution time: 128.504 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:04.543.203 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_2 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:04.543.379 [mindspore/ccsrc/runtime/device/kernel_runtime_manager.cc:35] ClearGraphResource] Clear device Ascend_3 graph 2 runtime resource [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.545.823 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:198] Compile] Input plan: +-Transfer,send_epoch_end:false,total_batch:0) | +-Repeat(count:1) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.545.966 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:216] Compile] Plan before optimization: +-Top | +-Transfer,send_epoch_end:false,total_batch:0) | | +-Repeat(count:1) | | | +-Batch(batch_size:32 drop_remainder:true) | | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | | +-MnistDataset [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.545.989 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:60] PrePass] Running pre pass loops. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.546.012 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:136] RunOnTree] Pre pass: node offload pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.546.055 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_offload_pass.cc:155] RunOnTree] Pre pass: offload node removal pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.546.147 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:59] RunOnTree] Pre pass: node removal pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.546.178 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/node_removal_pass.cc:73] RunOnTree] Pre pass: node removal pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.546.193 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:78] RunOnTree] Pre pass: Injection pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.546.216 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/epoch_ctrl_pass.cc:93] RunOnTree] Pre pass: Injection pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.546.229 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/cache_transform_pass.cc:182] RunOnTree] Pre pass: Cache transform pass started. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.546.252 [mindspore/ccsrc/minddata/dataset/engine/opt/pre/cache_transform_pass.cc:199] RunOnTree] Pre pass: Cache transform pass complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.546.264 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:91] PrePass] Pre pass offload complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.546.278 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:116] PostPass] Running post pass loops. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.546.310 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:135] PostPass] Post passes complete. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:04.546.358 [mindspore/ccsrc/minddata/dataset/engine/tree_adapter.cc:230] Compile] Plan after optimization: +-Top | +-Transfer,send_epoch_end:false,total_batch:0) | | +-Batch(batch_size:32 drop_remainder:true) | | | +-Shuffle(shuffle_size:10000,reset_every_epoch:true) | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | +-Map(,input:[image],output:[image],num_tensor_ops:1,...) | | | | | | | | +-Map(,input:[label],output:[label],num_tensor_ops:1,...) | | | | | | | | | +-MnistDataset [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:04.547.015 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_data_queue.cc:227] AscendTdtQueue] Select MBUF channel, the capacity of data queue is: 128 [INFO] MD(164046,fffd72ffd0f0,python):2024-01-10-11:41:04.549.745 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:254] WalkAllFiles] Mnist operator found image file at t10k-images-idx3-ubyte. [INFO] MD(164046,fffd72ffd0f0,python):2024-01-10-11:41:04.549.783 [mindspore/ccsrc/minddata/dataset/engine/datasetops/source/mnist_op.cc:257] WalkAllFiles] Mnist Operator found label file at t10k-labels-idx1-ubyte. [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.557.417 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:456] SendDataToAscend] Device queue, sending data to Ascend. [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.718.496 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:502] SendDataToAscend] Begin to send data to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.718.605 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:1182] PrintBeginInfoWhenFirstBatch] Loading dataset and begin to push first batch into device ... [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.719.128 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:1192] PrintEndInfoWhenFirstBatch] Loading dataset and push first batch into device successful. [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.719.151 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 1 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.719.665 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 2 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.720.103 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 3 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.720.544 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 4 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.721.024 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 5 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.721.656 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 6 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.722.152 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 7 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.722.560 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 8 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.722.968 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 9 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.723.380 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 10 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.723.796 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 11 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.724.218 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 12 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.724.634 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 13 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.725.185 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 14 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.725.584 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 15 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.726.069 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 16 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.726.467 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 17 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.726.879 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 18 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.727.286 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 19 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.727.681 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 20 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.728.102 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 21 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.728.522 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 22 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.728.922 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 23 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.729.413 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 24 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.729.817 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 25 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.730.228 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 26 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.730.643 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 27 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.731.052 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 28 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.731.452 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 29 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.731.877 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 30 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.732.281 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 31 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.732.697 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 32 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.733.175 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 33 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.733.579 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 34 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.734.020 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 35 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.734.446 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 36 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.734.880 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 37 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.735.295 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 38 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.735.698 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 39 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.736.084 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 40 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.736.477 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 41 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.736.894 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 42 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.737.429 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 43 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.737.846 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 44 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.738.242 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 45 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.738.652 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 46 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.739.072 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 47 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.739.463 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 48 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.739.852 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 49 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.740.243 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 50 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.740.651 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 51 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.741.138 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 52 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.741.529 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 53 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.742.001 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 54 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.742.394 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 55 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.742.802 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 56 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.743.203 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 57 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.743.598 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 58 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.744.014 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 59 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.744.416 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 60 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.744.801 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 61 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.745.300 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 62 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.745.762 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 63 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.746.156 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 64 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.746.556 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 65 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.746.927 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 66 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.747.317 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 67 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.747.678 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 68 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.748.065 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 69 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.748.453 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 70 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.748.858 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 71 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.749.328 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 72 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.749.768 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 73 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.750.166 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 74 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.750.571 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 75 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.750.978 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 76 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.751.366 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 77 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.751.763 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:541] SendDataToAscend] Have sent 78 batch(es) to device, channel name: 14b7bd8a-af6a-11ee-b370-30fd658829ca [INFO] MD(164046,fffd78ff90f0,python):2024-01-10-11:41:04.751.917 [mindspore/ccsrc/minddata/dataset/engine/datasetops/data_queue_op.cc:596] SendDataToAscend] ExecutionTree finished. Device queue sent number of batches: 78 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.903.538 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:978] CompileInner] Start compiling, phase: eval.1704858064751680000.281469723781744.0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.903.622 [mindspore/ccsrc/pipeline/jit/ps/event_message_print.cc:37] PrintEventMessage] Start compiling '_DataWrapper.construct' and it will take a while. Please wait... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.903.695 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1659] VmPipeline] This worker is initialized. No need to add worker action. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:04.903.732 [mindspore/ccsrc/backend/graph_compiler/transform.cc:573] CreateBackend] CreateBackend is: ge [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:04.903.752 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:04.903.768 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEBUG(164046,ffff82e63440,python):2024-01-10-11:41:04.903.981 [mindspore/ccsrc/debug/debugger/debugger.cc:129] Init] Debugger got device_id: 3 [INFO] DEBUG(164046,ffff82e63440,python):2024-01-10-11:41:04.903.998 [mindspore/ccsrc/debug/debugger/debugger.cc:131] Init] Debugger got device_target: Ascend [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.904.023 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1311] Run] Pipeline run [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.904.048 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start parse action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.905.961 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end parse action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.906.017 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start symbol_resolve action. [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.914.469 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380] Added global python symbol: {_check_is_tensor : Prim[_check_is_tensor]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.915.005 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_381{[0]: CNode_382, [1]: ValueNode logits, [2]: param_logits, [3]: CNode_383}, block: 0x4e0b5910/mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/nn/loss/loss.py:777/ _check_is_tensor('logits', logits, self.cls_name)/ [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.915.543 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_384{[0]: CNode_382, [1]: ValueNode labels, [2]: param_labels, [3]: CNode_385}, block: 0x4e0b5910/mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/nn/loss/loss.py:778/ _check_is_tensor('labels', labels, self.cls_name)/ [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.916.251 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_386{[0]: ValueNode Depend, [1]: CNode_387, [2]: CNode_388}, state: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_389{[0]: ValueNode MakeTuple, [1]: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_381{[0]: CNode_382, [1]: ValueNode logits, [2]: param_logits, [3]: CNode_383}, [2]: @mindspore_nn_loss_loss_SoftmaxCrossEntropyWithLogits_construct_380:CNode_384{[0]: CNode_382, [1]: ValueNode labels, [2]: param_labels, [3]: CNode_385}} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.918.541 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_391, [1]: param_x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.918.836 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_392, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.919.108 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_393, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.919.381 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_394, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.919.641 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_395, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.919.906 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_396, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.920.178 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_397, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.920.441 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_398, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.920.697 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_399, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.920.959 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_400, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.921.218 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_401, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.921.510 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] __main___LeNet5_construct_390 update var `x` with node @__main___LeNet5_construct_390:x{[0]: CNode_402, [1]: x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.924.100 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_403] Added global python symbol: {len : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.924.267 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.924.620 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.924.798 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.925.288 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_403:CNode_405{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.925.427 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_404:x{[0]: CNode_406, [1]: param_фx, [2]: CNode_405} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.925.924 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_403:CNode_407{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.926.372 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_404] Added global python symbol: {len : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.926.444 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_403:CNode_408{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.926.558 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_403:x_shape{[0]: CNode_409, [1]: param_x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.926.699 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.926.992 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.927.271 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_404] Added global python symbol: {F : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.927.366 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_403] Added global python symbol: {F : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.927.435 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_404 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_403:CNode_410{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.927.759 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.932.214 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_411] Added global python symbol: {len : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.932.379 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.932.707 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.932.874 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.933.324 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_411:CNode_413{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.933.462 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_412:x{[0]: CNode_414, [1]: param_фx, [2]: CNode_413} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.933.917 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_411:CNode_415{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.934.360 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_412] Added global python symbol: {len : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.934.427 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_411:CNode_416{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.934.537 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_411:x_shape{[0]: CNode_417, [1]: param_x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.934.672 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.934.952 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.935.212 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_412] Added global python symbol: {F : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.935.317 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_411] Added global python symbol: {F : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.935.372 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_412 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_411:CNode_418{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.935.673 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.939.733 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_419] Added global python symbol: {len : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.939.898 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.940.233 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.940.402 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.940.850 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `self.weight` with node @mindspore_nn_layer_basic_Dense_construct_419:CNode_421{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode weight} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.940.986 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `x` with node @↓mindspore_nn_layer_basic_Dense_construct_420:x{[0]: CNode_422, [1]: param_фx, [2]: CNode_421} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.941.399 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `self.bias` with node @mindspore_nn_layer_basic_Dense_construct_419:CNode_423{[0]: ValueNode resolve, [1]: ValueNode ClassMember: 'Namespace:mindspore.nn.layer.basic..', [2]: ValueNode bias} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.941.848 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_420] Added global python symbol: {len : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.941.916 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `len` with node @mindspore_nn_layer_basic_Dense_construct_419:CNode_424{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode len} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.942.020 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `x_shape` with node @mindspore_nn_layer_basic_Dense_construct_419:x_shape{[0]: CNode_425, [1]: param_x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.942.166 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:2 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.942.437 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.942.694 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_basic_Dense_construct_420] Added global python symbol: {F : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.942.786 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Dense_construct_419] Added global python symbol: {F : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.942.842 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_basic_Dense_construct_420 update var `F` with node @mindspore_nn_layer_basic_Dense_construct_419:CNode_426{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.basic', [2]: ValueNode F} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.943.141 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.945.239 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_basic_Flatten_construct_427] Added global python symbol: {F : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.948.138 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1261] ParseNameConstant] The NameConstant is bool:False [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.948.428 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:3 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.948.681 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.948.808 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1261] ParseNameConstant] The NameConstant is bool:True [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.949.522 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_pooling_MaxPool2d_construct_428] Added global python symbol: {isinstance : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.949.621 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_pooling_MaxPool2d_construct_429] Added global python symbol: {isinstance : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.949.677 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_pooling_MaxPool2d_construct_428 update var `isinstance` with node @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_430{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.pooling', [2]: ValueNode isinstance} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.949.860 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↓mindspore_nn_layer_pooling_MaxPool2d_construct_428] Added global python symbol: {tuple : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.949.944 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [mindspore_nn_layer_pooling_MaxPool2d_construct_429] Added global python symbol: {tuple : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.950.005 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↓mindspore_nn_layer_pooling_MaxPool2d_construct_428 update var `tuple` with node @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_431{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.nn.layer.pooling', [2]: ValueNode tuple} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.950.345 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.950.461 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.950.657 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.950.764 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.951.031 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.962.753 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.962.910 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.963.487 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @canonicalize_axis_432:CNode_433{[0]: ValueNode check_axis_valid_434, [1]: param_axis, [2]: ndim}, block: 0x4aab6b00/canonicalize_axis_432, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1606/ check_axis_valid(axis, ndim)/ [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.963.654 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.963.947 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @canonicalize_axis_432:CNode_435{[0]: ValueNode Depend, [1]: CNode_436, [2]: CNode_437}, state: @canonicalize_axis_432:CNode_433{[0]: ValueNode check_axis_valid_434, [1]: @canonicalize_axis_432:param_axis, [2]: @canonicalize_axis_432:ndim{[0]: CNode_438}} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.964.230 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {isinstance : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.964.377 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {Tensor : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.964.940 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {int : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.965.395 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {bool : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.966.107 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {check_flatten_order_const : Prim[check_flatten_order]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.966.600 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @2↓flatten_440:CNode_441{[0]: CNode_442, [1]: param_order}, block: 0x4a846f70/2↓flatten_440, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1615/ check_flatten_order_const(order)/ [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.967.031 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {rank_ : Prim[Rank]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.967.379 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.967.441 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.967.665 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {reshape_ : Prim[Reshape]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.967.849 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.968.147 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {ops : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.968.355 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.968.860 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {transpose_ : Prim[Transpose]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.969.274 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_443] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.969.383 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.969.450 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `shape_` with node @flatten_439:CNode_444{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode shape_} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.969.825 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_443] Added global python symbol: {rank_ : Prim[Rank]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.969.900 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `rank_` with node @flatten_439:CNode_445{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode rank_} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.970.196 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `start_dim` with node @flatten_439:param_start_dim [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.970.349 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.970.499 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `end_dim` with node @flatten_439:param_end_dim [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.970.609 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.970.880 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.970.953 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.971.184 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_443] Added global python symbol: {reshape_ : Prim[Reshape]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.971.253 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `reshape_` with node @flatten_439:CNode_446{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode reshape_} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.971.437 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.971.738 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [3↓flatten_443] Added global python symbol: {flatten_ : Prim[Flatten]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.971.845 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [flatten_439] Added global python symbol: {flatten_ : Prim[Flatten]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.971.912 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `flatten_` with node @flatten_439:CNode_447{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode flatten_} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.972.235 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `canonicalize_axis` with node ValueNode canonicalize_axis_432 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.972.667 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 3↓flatten_443 update var `check_dim_valid` with node ValueNode check_dim_valid_448 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.973.135 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @4↓flatten_449:CNode_450{[0]: ValueNode check_dim_valid_448, [1]: start_dim, [2]: end_dim}, block: 0x4ac11230/4↓flatten_449, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1636/ check_dim_valid(start_dim, end_dim)/ [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.973.372 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:0 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.973.426 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.973.749 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.974.211 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.974.748 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.975.323 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.975.761 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @2↓flatten_440:CNode_451{[0]: ValueNode Depend, [1]: CNode_452, [2]: CNode_453}, state: @2↓flatten_440:CNode_441{[0]: @flatten_439:CNode_442{[0]: ValueNode resolve, [1]: ValueNode SymbolStr: 'Namespace:mindspore.ops.function.array_func', [2]: ValueNode check_flatten_order_const}, [1]: @flatten_439:param_order} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.975.891 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @4↓flatten_449:CNode_454{[0]: ValueNode Depend, [1]: CNode_455, [2]: CNode_456}, state: @4↓flatten_449:CNode_450{[0]: ValueNode check_dim_valid_448, [1]: @4↓flatten_449:idx{[0]: ValueNode canonicalize_axis_432, [1]: param_start_dim, [2]: x_rank}, [2]: @4↓flatten_449:end_dim{[0]: ValueNode canonicalize_axis_432, [1]: param_end_dim, [2]: x_rank}} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.976.003 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.976.095 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1196] ParseNum] The Num is int64_t:1 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.977.223 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:174] CheckFuncReturn] Function must has 'return' statement, but missing in function 'flatten' at /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1557. FuncGraph: ↓check_dim_valid_457. We will add a 'return None' statement automatically. [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.977.390 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:174] CheckFuncReturn] Function must has 'return' statement, but missing in function 'flatten' at /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/function/array_func.py:1557. FuncGraph: ↓check_axis_valid_458. We will add a 'return None' statement automatically. [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:04.989.671 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [shape_459] Added global python symbol: {shape_ : Prim[Shape]} [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.997.492 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end symbol_resolve action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.997.546 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start graph_reusing action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.997.565 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.basic.DenseDense[True, None]_ID [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.997.577 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.conv.Conv2dConv2dodict_values([6, 16, 5, 1, 'valid', 0, False, ])_ID [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.997.588 [mindspore/ccsrc/pipeline/jit/ps/action.cc:598] GraphReusingAction] Finish handling the reusable graph: mindspore.nn.layer.conv.Conv2dConv2dodict_values([1, 6, 5, 1, 'valid', 0, False, ])_ID [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.997.606 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end graph_reusing action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.997.623 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start meta_unpack_prepare action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.998.385 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end meta_unpack_prepare action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.998.424 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pre_cconv action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.998.439 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pre_cconv action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:04.998.468 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start abstract_specialize action. [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:04.999.934 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_463{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:04.999.995 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.000.408 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_464{[0]: CNode_465}, new_node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_466{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.000.461 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_464{[0]: CNode_465}, new node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_464{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.001.564 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_equal_scalar_467] Added global python symbol: {F : } [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.001.941 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 4, Shape: NoShape), AbstractScalar(Type: Int64, Value: 3, Shape: NoShape)}, g: _equal_scalar_467 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.002.599 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_468:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_468:CNode_470{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.002.667 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_468:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_468:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.004.782 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_472{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.004.841 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.005.181 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_473{[0]: CNode_474}, new_node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_475{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.005.233 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_473{[0]: CNode_474}, new node: @mindspore_nn_layer_pooling_MaxPool2d_construct_429:CNode_473{[0]: ValueNode PrimFunc_Rank, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.005.903 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_476:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_476:CNode_477{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.005.972 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_476:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_476:CNode_469{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.011.639 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_logical_not_scala_478] Added global python symbol: {auto_generate : } [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.012.070 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Bool, Value: true, Shape: NoShape)}, g: _logical_not_scala_478 [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.013.669 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bool_not_479] Added global python symbol: {_get_cache_prim : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.013.838 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [bool_not_479] Added global python symbol: {BoolNot : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.017.360 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_480] Added global python symbol: {str : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.017.815 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↻_get_cache_prim_for_pynative_481] Added global python symbol: {str : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.018.081 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] ↻_get_cache_prim_for_pynative_481 update var `str` with node @↵_get_cache_prim_for_pynative_482:param_фstr [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.018.305 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_480] Added global python symbol: {tuple : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.018.499 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] _get_cache_prim_for_pynative_483 update var `key` with node @_get_cache_prim_for_pynative_483:key{[0]: CNode_484, [1]: key, [2]: CNode_485} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.019.266 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_486] Added global python symbol: {str : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.019.846 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_486] Added global python symbol: {_PRIM_CACHE : {}} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.019.939 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_480] Added global python symbol: {_PRIM_CACHE : {}} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.020.182 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [↵_get_cache_prim_for_pynative_486] Added global python symbol: {Primitive : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.020.268 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_get_cache_prim_480] Added global python symbol: {Primitive : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.020.931 [mindspore/ccsrc/pipeline/jit/ps/parse/parse.cc:1003] ParseExpr] Isolated node found(NoReturn), no_return_node: @✓↓_get_cache_prim_for_pynative_487:CNode_488{[0]: ValueNode MetaFuncGraph-unpack_call.489, [1]: CNode_490, [2]: param_фargs, [3]: param_фkwargs}, block: 0x4dae7330/✓↓_get_cache_prim_for_pynative_487, Line: In file /home/jenkins/.local/lib/python3.7/site-packages/mindspore/ops/_primitive_cache.py:84/ prim.__init__(*args, **kwargs)/ [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.021.507 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] 2↓_get_cache_prim_for_pynative_491 update var `key` with node @↓_get_cache_prim_for_pynative_492:key{[0]: param_фstr, [1]: param_фkey} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.021.654 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @↻_get_cache_prim_for_pynative_493:CNode_494{[0]: ValueNode Depend, [1]: CNode_495, [2]: CNode_496}, state: @↻_get_cache_prim_for_pynative_493:CNode_497{[0]: ValueNode MetaFuncGraph-add.144, [1]: @↵_get_cache_prim_for_pynative_486:param_@CNode_497, [2]: ValueNode 1} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.021.778 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:860] AttachIsolatedNodesBeforeReturn] Attached for side-effect nodes, depend_node: @✓↓_get_cache_prim_for_pynative_487:CNode_498{[0]: ValueNode Depend, [1]: CNode_499, [2]: CNode_500}, state: @✓↓_get_cache_prim_for_pynative_487:CNode_488{[0]: ValueNode MetaFuncGraph-unpack_call.489, [1]: @✓↓_get_cache_prim_for_pynative_487:CNode_490{[0]: ValueNode getattr, [1]: prim, [2]: ValueNode __init__}, [2]: @↵_get_cache_prim_for_pynative_486:param_фargs, [3]: @↵_get_cache_prim_for_pynative_486:param_фkwargs} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.023.059 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_501:CNode_502{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.023.125 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_501:CNode_503{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.023.183 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2858] EvalPrim] For MakeTuple, the inputs should not be empty. node: @_new_prim_for_graph_501:CNode_504{[0]: ValueNode MakeTuple} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.023.775 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BoolNot. node: @bool_not_479:CNode_505{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x}, new_node: @bool_not_479:CNode_506{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.023.831 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BoolNot. node: @bool_not_479:CNode_505{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x}, new node: @bool_not_479:CNode_505{[0]: ValueNode PrimFunc_BoolNot, [1]: param_x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.028.099 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_equal_string_507] Added global python symbol: {F : } [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.028.434 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: String, Value: C, Shape: NoShape), AbstractScalar(Type: String, Value: F, Shape: NoShape)}, g: _equal_string_507 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.029.950 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_508:CNode_509{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_508:CNode_510{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.030.019 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_508:CNode_509{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_508:CNode_509{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.031.251 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_neg_scalar_511] Added global python symbol: {F : } [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.031.561 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 1, Shape: NoShape)}, g: _neg_scalar_511 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.032.154 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarUsub. node: @_neg_scalar_512:CNode_513{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x}, new_node: @_neg_scalar_512:CNode_514{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.032.215 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarUsub. node: @_neg_scalar_512:CNode_513{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x}, new node: @_neg_scalar_512:CNode_513{[0]: ValueNode PrimFunc_ScalarUsub, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.032.866 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_equal_scalar_515:CNode_516{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_equal_scalar_515:CNode_517{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.032.946 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_equal_scalar_515:CNode_516{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_equal_scalar_515:CNode_516{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.033.375 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Rank. node: @3↓flatten_518:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput}, new_node: @3↓flatten_518:CNode_519{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.033.439 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Rank. node: @3↓flatten_518:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput}, new node: @3↓flatten_518:x_rank{[0]: ValueNode PrimFunc_Rank, [1]: param_фinput} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.034.822 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_520] Added global python symbol: {F : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.035.477 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_520] Added global python symbol: {InSequence : } [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.035.822 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_number_in_tuple_520] Added global python symbol: {const_utils : } [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.036.295 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 4, Shape: NoShape), AbstractTuple{ element[0]: AbstractScalar(Type: Int64, Value: 0, Shape: NoShape), element[1]: AbstractScalar(Type: Int64, Value: 1, Shape: NoShape), sequence_nodes: {@✓3↓flatten_521:CNode_522{[0]: ValueNode MakeTuple, [1]: ValueNode 0, [2]: ValueNode 1}, elements_use_flags: {ptr: 0x4abe33a0, value: [const vector]{0, 0}}}, dynamic_len:0, is dyn arg:0} }, g: _number_in_tuple_520 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.039.637 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Flatten. node: @↓✓3↓flatten_523:CNode_524{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput}, new_node: @↓✓3↓flatten_523:CNode_525{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.039.707 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Flatten. node: @↓✓3↓flatten_523:CNode_524{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput}, new node: @↓✓3↓flatten_523:CNode_524{[0]: ValueNode PrimFunc_Flatten, [1]: param_фinput} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.040.013 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_419:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_419:CNode_526{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.040.077 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_419:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_419:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.041.299 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [_not_equal_scalar_527] Added global python symbol: {F : } [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.041.644 [mindspore/core/ir/meta_func_graph.cc:71] GenerateFuncGraph] MetaFuncgraph: cache miss for types: [const vector]{AbstractScalar(Type: Int64, Value: 2, Shape: NoShape), AbstractScalar(Type: Int64, Value: 2, Shape: NoShape)}, g: _not_equal_scalar_527 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.042.401 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_528:CNode_529{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_528:CNode_530{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.042.470 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_528:CNode_529{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_528:CNode_529{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.044.395 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_532:CNode_533{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_532:CNode_534{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.044.464 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_532:CNode_533{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_532:CNode_533{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.045.627 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_535:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_535:CNode_536{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias, [3]: ValueNode 0} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.045.730 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_535:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_535:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc1.bias, [3]: ValueNode 0} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.045.988 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_537{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.046.051 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.046.219 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_411:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_411:CNode_538{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.046.264 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_411:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_411:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.047.121 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_539:CNode_540{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_539:CNode_541{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.047.188 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_539:CNode_540{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_539:CNode_540{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.048.948 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_542:CNode_543{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_542:CNode_544{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.049.015 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_542:CNode_543{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_542:CNode_543{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.050.209 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_545:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_545:CNode_546{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias, [3]: ValueNode 0} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.050.278 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_545:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_545:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc2.bias, [3]: ValueNode 0} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.050.537 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new_node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_547{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.050.587 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ReLU. node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x}, new node: @mindspore_nn_layer_activation_ReLU_construct_461:CNode_462{[0]: ValueNode PrimFunc_ReLU, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.050.753 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: Shape. node: @mindspore_nn_layer_basic_Dense_construct_403:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new_node: @mindspore_nn_layer_basic_Dense_construct_403:CNode_548{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.050.798 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_Shape. node: @mindspore_nn_layer_basic_Dense_construct_403:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x}, new node: @mindspore_nn_layer_basic_Dense_construct_403:x_shape{[0]: ValueNode PrimFunc_Shape, [1]: param_x} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.051.635 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_549:CNode_550{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_549:CNode_551{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.051.711 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_549:CNode_550{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_549:CNode_550{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.053.474 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: ScalarEq. node: @_not_equal_scalar_552:CNode_553{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new_node: @_not_equal_scalar_552:CNode_554{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.053.542 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_ScalarEq. node: @_not_equal_scalar_552:CNode_553{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y}, new node: @_not_equal_scalar_552:CNode_553{[0]: ValueNode PrimFunc_ScalarEq, [1]: param_x, [2]: param_y} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.054.730 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2608] CheckAndConvertPrimitiveArgs] Convert primitive args: BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_555:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias}, new_node: @✓↓mindspore_nn_layer_basic_Dense_construct_555:CNode_556{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias, [3]: ValueNode 0} [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.054.812 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/prim.cc:2686] EvalPrim] Convert primitive args to inputs: PrimFunc_BiasAdd. node: @✓↓mindspore_nn_layer_basic_Dense_construct_555:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias}, new node: @✓↓mindspore_nn_layer_basic_Dense_construct_555:x{[0]: ValueNode PrimFunc_BiasAdd, [1]: x, [2]: param_fc3.bias, [3]: ValueNode 0} [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.055.646 [mindspore/ccsrc/pipeline/jit/ps/action.cc:282] AbstractAnalyze] function call depth: 0, simulate call depth: 0 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.055.776 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:222] Run] Specialize set top func graph context: {FuncGraph: mindspore_train_dataset_helper__DataWrapper_construct_460 Args: [0]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [1]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [2]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [3]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [4]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [5]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [6]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [7]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), Parent: } [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.063.003 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end abstract_specialize action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.063.059 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pack_expand action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.063.176 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pack_expand action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.063.206 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start auto_monad action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.064.187 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end auto_monad action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.064.234 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start inline action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.064.255 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end inline action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.064.293 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pre_auto_parallel action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.064.318 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pre_auto_parallel action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.064.336 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start pipeline_split action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.064.351 [mindspore/ccsrc/pipeline/jit/ps/pipeline_split.cc:247] PipelineSplit] Only auto_parallel and semi_auto_parallel support pipeline split. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.064.362 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end pipeline_split action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.064.378 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start optimize action. [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.067.267 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [fill_557] Added global python symbol: {cast_ : Prim[Cast]} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.067.495 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:109] WriteVariable] fill_557 update var `value` with node @fill_557:value{[0]: CNode_558, [1]: param_value, [2]: param_type} [INFO] PARSER(164046,ffff82e63440,python):2024-01-10-11:41:05.067.720 [mindspore/ccsrc/pipeline/jit/ps/parse/function_block.cc:366] HandleNamespaceSymbol] [fill_557] Added global python symbol: {fillv2_ : Prim[FillV2]} [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:41:05.073.389 [mindspore/ccsrc/frontend/parallel/pynative_shard/pynative_shard.cc:334] PynativeShard] Only auto_parallel and semi_auto_parallel support pynative shard [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:41:05.073.451 [mindspore/ccsrc/frontend/parallel/step_parallel.cc:3009] StepParallel] Strategies would be ignored in data_parallel, shard() only valid in [semi_]auto_parallel. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.075.902 [mindspore/ccsrc/pipeline/jit/ps/action.cc:282] AbstractAnalyze] function call depth: 0, simulate call depth: 0 [INFO] ANALYZER(164046,ffff82e63440,python):2024-01-10-11:41:05.076.711 [mindspore/ccsrc/pipeline/jit/ps/static_analysis/program_specialize.cc:222] Run] Specialize set top func graph context: {FuncGraph: 382_mindspore_train_dataset_helper__DataWrapper_construct_559 Args: [0]: AbstractRefTensor(key: conv2.weight ref_value: AbstractRefTensor(shape: (16, 6, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [1]: AbstractRefTensor(key: conv1.weight ref_value: AbstractRefTensor(shape: (6, 1, 5, 5), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [2]: AbstractRefTensor(key: fc3.bias ref_value: AbstractRefTensor(shape: (10), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [3]: AbstractRefTensor(key: fc3.weight ref_value: AbstractRefTensor(shape: (10, 84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [4]: AbstractRefTensor(key: fc2.bias ref_value: AbstractRefTensor(shape: (84), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [5]: AbstractRefTensor(key: fc2.weight ref_value: AbstractRefTensor(shape: (84, 120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [6]: AbstractRefTensor(key: fc1.bias ref_value: AbstractRefTensor(shape: (120), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), [7]: AbstractRefTensor(key: fc1.weight ref_value: AbstractRefTensor(shape: (120, 400), element: AbstractScalar(Type: Float32, Value: ValueAny, Shape: NoShape), value_ptr: 0x2ab272b0, value: ValueAny), value: ValueAny), Parent: } [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:41:05.079.219 [mindspore/ccsrc/frontend/parallel/pynative_shard/pynative_shard.cc:334] PynativeShard] Only auto_parallel and semi_auto_parallel support pynative shard [INFO] OPTIMIZER(164046,ffff82e63440,python):2024-01-10-11:41:05.081.276 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] OPTIMIZER(164046,ffff82e63440,python):2024-01-10-11:41:05.081.713 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] OPTIMIZER(164046,ffff82e63440,python):2024-01-10-11:41:05.082.036 [mindspore/ccsrc/frontend/optimizer/optimizer.h:221] operator()] Optimizer::step: Skipping Renormalize because is_untyped_generated_ is False. [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:41:05.082.112 [mindspore/ccsrc/frontend/parallel/cache_embedding/cache_embedding.cc:702] AddCacheEmbedding] Parameters are all not cache enable. [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:41:05.082.519 [mindspore/ccsrc/frontend/parallel/pass/assign_add_opt.cc:120] AssignAddOpt] parallel::g_device_manager is not initialized. [INFO] OPTIMIZER(164046,ffff82e63440,python):2024-01-10-11:41:05.082.578 [mindspore/ccsrc/frontend/optimizer/comm_op_reuse_tag.cc:59] AddCommOpReuseTag] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:41:05.082.601 [mindspore/ccsrc/frontend/parallel/pass/overlap_opt_shard_in_pipeline.cc:70] OverlapOptShardInPipeline] parallel::g_device_manager is not initialized. [INFO] OPTIMIZER(164046,ffff82e63440,python):2024-01-10-11:41:05.082.618 [mindspore/ccsrc/frontend/optimizer/grouped_pairwise_exchange_alltoall.cc:673] SetGroupedPairwiseExchangeAllToAll] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:41:05.082.639 [mindspore/ccsrc/frontend/parallel/pass/overlap_gradmatmul_and_gradallreduce.cc:358] OverlapGradMatmulAndGradAllreduce] parallel::g_device_manager is not initialized. [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:41:05.082.657 [mindspore/ccsrc/frontend/parallel/pass/split_matmul_comm_elementwise_fp.cc:184] SplitMatmulCommElementwiseFp] SplitMatmulCommElementwiseFp is only support under [semi_]auto_parallel, skip it. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.082.683 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end optimize action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.082.701 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start auto_monad_reorder action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.082.821 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end auto_monad_reorder action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.082.848 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start get_jit_bprop_graph action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.082.860 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end get_jit_bprop_graph action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.082.879 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start eliminate_special_op_node action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.083.474 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end eliminate_special_op_node action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.083.533 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start validate action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.083.644 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end validate action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.083.671 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start distribtued_split action. [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:41:05.083.689 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:372] GenerateStrategy] Current parallel mode is data_parallel [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:41:05.083.704 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:384] GenerateStrategy] Generated distributed strategy is 1 [INFO] PARALLEL(164046,ffff82e63440,python):2024-01-10-11:41:05.083.825 [mindspore/ccsrc/frontend/parallel/graph_util/graph_splitter.cc:1270] Run] All nodes are on this precoess so there's no need to build and split distributed graph. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.083.846 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end distribtued_split action. [INFO] PROFILER(164046,ffff82e63440,python):2024-01-10-11:41:05.083.877 [mindspore/ccsrc/plugin/device/ascend/hal/profiler/parallel_strategy_profiling.cc:48] IsProfilingParallelStrategyEnabled] Profiling parallel strategy is disabled. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.083.898 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start task_emit action. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.031 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.050 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.062 [mindspore/ccsrc/pipeline/jit/ps/action.cc:1252] SetRunMode] Run graph mode with kernel by kernel by configuration. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.084.102 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:546] CompileGraphs] Status record: start compile function graph: 458_382_mindspore_train_dataset_helper__DataWrapper_construct_560 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.084.210 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unify_mindir_pm_0_erase_invalid_micro_depend in 1.57 us [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.585 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.609 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.729 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.747 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.769 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.781 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.800 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.811 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.837 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.852 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.867 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.882 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.897 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.908 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.922 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.935 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.949 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.961 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.977 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.084.990 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.005 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.019 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.033 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.047 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.095 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.108 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.123 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.138 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.156 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.170 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.188 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.208 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.224 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.240 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.255 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.267 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.281 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.292 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.309 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.324 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.339 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.349 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.364 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.376 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.390 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.401 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.415 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.426 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.440 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.454 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.471 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.482 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.499 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.510 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.523 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.540 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.555 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.566 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.584 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.595 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.610 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.624 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.639 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.654 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.667 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.085.681 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.085.788 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:727] CompileGraph] Compile graph: 458_382_mindspore_train_dataset_helper__DataWrapper_construct_560, Split segments size: 2 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.085.830 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:762] CompileGraphFromSegment] Compile normal segment, the first node: @458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:CNode_561{[0]: ValueNode Load, [1]: param_fc3.bias, [2]: ValueNode U} [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.085.996 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:421] CompileGraph] Status record: start compile graph. [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.086.030 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:1940] ConstructKernelGraph] Create graph: 3 [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.086.608 [mindspore/ccsrc/backend/common/session/kernel_graph_mgr.cc:2697] ConstructOutput] Output:@458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:562{[0]: ValueNode Depend, [1]: 562, [2]: CNode_563} [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.087.538 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:193] EliminateIllegalDataTypePass] Start eliminate illegal data type for kernel graph id:3 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.088.022 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_0_convert_list_to_tuple in 32.69 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.088.278 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_eliminate_illegal_data_type_pm_1_eliminate_func_type in 208.14 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.088.680 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:154] CommonUnifyMindIR] start common unify mindir opt graph:3 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.089.094 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: conv_transpose_to_conv_backprop_input [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.089.275 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_0_conv_transpose_to_conv_backprop_input in 177.24 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.089.305 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: custom_op_reg_info_to_attr [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.089.339 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_1_custom_op_reg_info_to_attr in 30.88 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.089.356 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim Custom not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.089.371 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_2_inplace_assign_for_custom_op in 13.41 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.089.387 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_attr_to_unify_mindir [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.089.566 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_unify_mindir_pm_3_convert_attr_to_unify_mindir in 171.26 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.090.031 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:79] BackendCommonOptimization] Status record: start common optimization. graph id: 3 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.090.431 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: add_dynamic_shape_attr [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.090.711 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_0_add_dynamic_shape_attr in 272.74 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.090.742 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: convert_dynamic_broadcast_to [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.090.789 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_1_convert_dynamic_broadcast_to in 42.45 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.090.988 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_2_convert_const_input_to_attr in 174.53 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.091.158 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_3_custom_op_const_input_to_attr in 140.4 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.091.306 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_4_convert_const_input_to_tensor_input_for_print in 120.92 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.091.735 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_5_convert_tuple_output_to_maketuple in 398.23 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.091.773 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_6_convert_unused_tuple_para_to_make_tuple in 7.19 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.091.913 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_7_flatten_concat_fission in 115.74 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.092.049 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_8_inset_input_structural_for_py_execute in 107.35 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.092.195 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_9_broadcast_to_fusion in 106.59 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.092.362 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_common_pm_10_add_attr_to_node in 140.96 us [WARNING] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.092.769 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:92] BackendCommonOptimization] [PROF]backend_common_optimization costs 2739 usec. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.092.801 [mindspore/ccsrc/backend/common/optimizer/common_backend_optimization.cc:99] BackendCommonOptimization] Status record: end common optimization. graph id: 3 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.093.409 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_0_renorm_split in 133.69 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.093.442 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: reduce_axis_update [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.093.775 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_1_reduce_axis_update in 322.26 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.093.805 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim HistogramFixedWidth not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.093.825 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_2_histogram_fixed_width_fusion in 19.87 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.093.840 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim ClipByNorm not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.093.854 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_3_clip_by_norm_fission in 13.81 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.093.869 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.093.885 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_4_tensor_arry_add_flow_cond1 in 16.06 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.093.898 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayGather not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.093.913 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_5_tensor_arry_add_flow_cond2 in 13.77 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.093.926 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArrayWrite not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.093.941 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_6_ge_tensor_arry_cast_index in 13.52 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.093.956 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim TensorArray not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.093.972 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_7_tensor_array_prepare in 14.53 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.093.987 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: space_to_batch_nd_attr_update [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.094.035 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_8_space_to_batch_nd_attr_update in 43.7 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.094.054 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: batch_to_space_nd_attr_update [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.094.082 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_9_batch_to_space_nd_attr_update in 26.18 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.094.101 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: avg_pool_grad_unify_mindir [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.094.134 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_10_avg_pool_grad_unify_mindir in 31.6 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.094.281 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_11_adam_weight_decay_unify_mindir in 123.77 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.094.418 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_12_cdist_fission in 110.77 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.094.552 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_13_cdist_grad_fission in 108.13 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.094.714 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_14_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir in 134.68 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.094.880 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_15_grad_sparse_softmax_cross_entropy_with_logits_unify_mindir_v2 in 138.24 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.095.530 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_16_sparse_softmax_cross_entropy_with_logits_unify_mindir in 608.59 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.095.719 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_17_dropout_unify_mindir1 in 142.93 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.095.748 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: dropoutgrad_unify_mindir [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.095.904 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_18_dropoutgrad_unify_mindir in 149.78 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.095.930 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchange not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.095.950 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_19_neighbor_exchange_unify_mindir in 18.92 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.095.965 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2 not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.095.980 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_20_neighbor_exchange_v2_unify_mindir in 13.46 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.095.997 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim NeighborExchangeV2Grad not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.022 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_21_neighbor_exchange_v2_grad_unify_mindir in 23.01 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.039 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim AlltoAll not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.053 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_22_all_to_all_unify_mindir in 12.6 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.072 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim QuantDTypeCast not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.086 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_23_quant_dtype_cst_adjust in 15.05 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.101 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim FSEDecode not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.118 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_24_fse_decode_adjust in 16.34 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.131 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.147 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_25_bn_split in 14.82 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.159 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: bn_grad_unify_mindir [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.215 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_26_bn_grad_unify_mindir in 52.63 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.235 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.250 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_27_bn_grad_split in 14.07 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.263 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNorm not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.279 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_28_batchnorm_to_bninfer in 14.88 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.292 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.306 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_29_batchnormgrad_to_bninfergrad in 11.84 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.322 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:387] Run] Prim BatchNormGrad not exist in name to cnode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.336 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_30_batch_norm_grad_infer_fission in 12.75 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.600 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_31_ascend_mindir_op_adapter in 242.19 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.630 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_32_DropoutGenMask in 0.97 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.096.811 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_33_lamb_fission_ge in 141.23 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.097.135 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_34_print_insert_placeholder_for_tensor_name in 294.35 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.097.321 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_35_getnext_for_ge in 156.46 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.097.476 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_36_sync_bn_split in 128.07 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.097.627 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_37_sync_bn_grad_split in 125.24 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.097.801 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_38_adaptive_max_pool2d_ge_fusion in 146.15 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.097.960 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_39_avg_pool_grad_for_ge in 127.45 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.097.988 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:260] DefineFlashAttentionPattern] Do FlashAttentionPattern V1. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.098.360 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_40_FlashAttentionFusionV1 in 367.92 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.098.388 [mindspore/ccsrc/plugin/device/ascend/optimizer/ir_fusion/flash_attention_fusion.cc:374] DefineFlashAttentionPattern] Do FlashAttentionPattern V2. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.098.754 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_ge_unify_mindir_pm_41_FlashAttentionFusionV1 in 361.5 us [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.100.041 [mindspore/core/utils/ms_context.cc:498] IsKByKExecutorMode] The jit level is: O1 [INFO] CORE(164046,ffff82e63440,python):2024-01-10-11:41:05.100.076 [mindspore/core/utils/ms_context.cc:517] IsKByKExecutorMode] The graph mode enable kbyk executor mode by GRAPH_OP_RUN or JitLevelO0. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:05.100.092 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_device_context.cc:132] GetRunMode] RunMode::kKernelMode [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.100.554 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_unfold_inputs_for_special_nodes_pm_0_ascend_convert_tuple_input_to_dynamic_input in 417.12 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.101.215 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_0_seed_adapter in 395.14 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.101.247 [mindspore/ccsrc/backend/common/optimizer/node_pass.cc:378] Run] Run fast pass: env_op_attr_update [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.101.492 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_1_env_op_attr_update in 237.06 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.101.658 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_2_process call inline in 135.51 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.101.805 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_pm_3_expander_fallback in 117.42 us [WARNING] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.102.048 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:139] GEBackendOptimizeACL] [PROF]ascend_backend_optimize_acl costs 1244 usec. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.102.094 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] GetNext is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.102.174 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2D is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.102.343 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPool is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.102.416 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] Conv2D is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.102.538 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MaxPool is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.102.648 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.102.861 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.103.036 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] MatMul is not defined in opdef. [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.103.382 [mindspore/ccsrc/plugin/device/ascend/kernel/opapi/aclnn_kernel_build.cc:73] IsEnabledAclnnDispatch] SoftmaxCrossEntropyWithLogits is not defined in opdef. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.103.792 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_0_set_fracz_group_attr in 57.36 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.104.166 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_1_insert_identity in 328.4 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.104.531 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_2_erase_visit_attr in 326.97 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.105.062 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_3_deal_ref_output in 491.43 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.105.532 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_opt_acl_after_kernel_select_pm_4_insert_type_transform_op in 429.45 us [WARNING] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.105.819 [mindspore/ccsrc/plugin/device/ascend/optimizer/ge_backend_optimization.cc:179] GEBackendOptimizeACLAfterKernelSelect] [PROF]ascend_backend_optimize_acl_after_kernel_select costs 2128 usec. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.105.901 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_0_DropoutGenMask in 0.69 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.106.063 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_1_cse in 128.76 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.106.859 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_2_eliminate_redundant_op in 764.19 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.107.038 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_3_eliminate_maketuple_getitem in 131.85 us [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.107.068 [mindspore/ccsrc/backend/common/optimizer/pass_manager.cc:40] RunPass] Run pass hwopt_after_inline_pm_4_MergeTransData in 0.86 us [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.107.449 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive GetNext [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:41:05.107.636 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:467] ConvertAny] Value: ValueTuple [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.107.728 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive GetNext [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.107.754 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2D [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:41:05.107.827 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.107.918 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2D [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.107.941 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.108.010 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.108.032 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPool [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.108.120 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPool [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.108.143 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Conv2D [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:41:05.108.193 [mindspore/ccsrc/transform/graph_ir/op_adapter.h:548] ConvertAny] Value: ValueTuple [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.108.267 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Conv2D [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.108.288 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.108.351 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.108.372 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MaxPool [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.108.451 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MaxPool [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.108.472 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Flatten [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.108.549 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Flatten [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.108.573 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.108.659 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.108.682 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.108.793 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.108.817 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.108.880 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.108.901 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.108.974 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.108.994 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.109.075 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.109.099 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReLU [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.109.159 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReLU [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.109.182 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.109.255 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive MatMul [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.109.279 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive BiasAdd [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.109.363 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive BiasAdd [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.109.387 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.109.463 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.109.486 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive Reshape [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.109.578 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive Reshape [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.109.602 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive OneHot [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.109.776 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive OneHot [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.109.805 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive SoftmaxCrossEntropyWithLogits [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.109.907 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive SoftmaxCrossEntropyWithLogits [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.109.932 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:30] AclOpBuild] Begin to create acl kernel module for primitive ReduceMean [INFO] KERNEL(164046,ffff82e63440,python):2024-01-10-11:41:05.110.033 [mindspore/ccsrc/plugin/device/ascend/kernel/acl/acl_kernel_build.cc:80] AclOpBuild] Finished creating acl kernel module for primitive ReduceMean [WARNING] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:05.110.055 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:262] CreateKernel] [PROF]create_kernel costs 2626 usec. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.110.187 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1_6, index: 0 to input Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2_6, index: 0 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.110.219 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/Reshape-op0_6, index: 0 to input Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6, index: 0 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.110.244 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:195] OptimizeNopNode] The reference relation of nopnode Default/Reshape-op1_6, index: 0 to input Default/GetNext-op1_6, index: 1 [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.110.259 [mindspore/ccsrc/backend/common/session/session_basic.cc:1140] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 3 start [INFO] DEBUG(164046,ffff82e63440,python):2024-01-10-11:41:05.110.271 [mindspore/ccsrc/debug/summary/summary.cc:33] RecurseSetSummaryNodesForAllGraphs] Recurse set summary nodes for all graphs in graph: 3 start [INFO] DEBUG(164046,ffff82e63440,python):2024-01-10-11:41:05.110.283 [mindspore/ccsrc/debug/summary/summary.cc:54] RecurseSetSummaryNodesForAllGraphs] The total summary nodes is: 0 for graph: 3 [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.110.497 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:891] PrintGraphExecuteOrder] Graph 3 execution order: [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.110.540 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[0], node name[Default/GetNext-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:outputs{[0]: ValueNode GetNext}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.110.587 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[1], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv1-Conv2d/Conv2D-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode Conv2D, [1]: 562, [2]: CNode_564}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.110.634 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[2], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op4_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReLU, [1]: 562}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.110.671 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[3], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op3_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode MaxPool, [1]: 562}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.110.710 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[4], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/conv2-Conv2d/Conv2D-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode Conv2D, [1]: 562, [2]: CNode_565}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.110.743 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[5], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op5_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReLU, [1]: 562}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.110.773 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[6], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode MaxPool, [1]: 562}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.110.811 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[7], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_Flatten, [1]: 562}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.110.852 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[8], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/MatMul-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode MatMul, [1]: 562, [2]: CNode_566}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.110.893 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[9], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc1-Dense/BiasAdd-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_BiasAdd, [1]: 562, [2]: CNode_567, [3]: ValueNode 0}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.110.927 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[10], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op6_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReLU, [1]: 562}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.110.963 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[11], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/MatMul-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode MatMul, [1]: 562, [2]: CNode_568}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.111.011 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[12], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc2-Dense/BiasAdd-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_BiasAdd, [1]: 562, [2]: CNode_569, [3]: ValueNode 0}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.111.045 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[13], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/relu-ReLU/ReLU-op7_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReLU, [1]: 562}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.111.082 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[14], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/MatMul-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode MatMul, [1]: 562, [2]: CNode_570}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.111.117 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[15], node name[Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_BiasAdd, [1]: 562, [2]: CNode_561, [3]: ValueNode 0}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.111.164 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[16], node name[Default/Reshape-op0_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_Reshape, [1]: 562, [2]: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10])}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.111.202 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[17], node name[Default/Reshape-op1_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_Reshape, [1]: 562, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[32])}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.111.260 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[18], node name[Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/OneHot-op0_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_OneHot, [1]: 562, [2]: ValueNode Tensor(shape=[], dtype=Int64, value=10), [3]: ValueNode Tensor(shape=[], dtype=Float32, value=1), [4]: ValueNode Tensor(shape=[], dtype=Float32, value=0), [5]: ValueNode -1}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.111.303 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[19], node name[Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/SoftmaxCrossEntropyWithLogits-op0_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode SoftmaxCrossEntropyWithLogits, [1]: 562, [2]: 562}] [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.111.347 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:931] PrintGraphExecuteOrder] Index[20], node name[Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/ReduceMean-op0_6], logic id[4294967295], stream id[0], node info[@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReduceMean, [1]: 562, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), [3]: ValueNode false}] [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.111.480 [mindspore/ccsrc/backend/common/somas/somas.cc:143] Assign] Start Somas Assign for graph 3 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.111.587 [mindspore/ccsrc/backend/common/somas/somas.cc:151] Assign] Somas Configure success, configuration info: Device Name: Ascend Run by execution order: 1 Enable debug log: 0 Debug log path: . [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.111.601 [mindspore/ccsrc/backend/common/somas/somas.cc:154] Assign] Start Initialize SOMAS Model [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.027 [mindspore/ccsrc/backend/common/somas/somas.cc:1065] GraphOutputProcess] Set 3 graph output tensors' aligned size to 0. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.199 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 7 output 8 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.227 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 16 output 17 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.242 [mindspore/ccsrc/backend/common/somas/somas.cc:1130] RefNodeProcess] RefNode: input 1 output 18 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.256 [mindspore/ccsrc/backend/common/somas/somas.cc:1135] RefNodeProcess] Special Tensor total size: RefNode: input 53760 output 52608 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.277 [mindspore/ccsrc/backend/common/somas/somas.cc:555] InitSomasModel] Created 1 streams (0 groups), 21 nodes, 23 tensors, 3 union tensors lists, and 0 contiguous tensors lists [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.434 [mindspore/ccsrc/backend/common/somas/somas.cc:157] Assign] End Initialize SOMAS Model [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.448 [mindspore/ccsrc/backend/common/somas/somas.cc:176] Assign] Start Computing Conflict Matrix [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.460 [mindspore/ccsrc/backend/common/somas/somas.cc:1286] ComputeBasicMatrix] Start Conflict Computing (Bitset Model) [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.473 [mindspore/ccsrc/backend/common/somas/somas.cc:1291] ComputeBasicMatrix] Start Bitset [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.490 [mindspore/ccsrc/backend/common/somas/somas.cc:1299] ComputeBasicMatrix] Start Path Computing [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.502 [mindspore/ccsrc/backend/common/somas/somas.cc:1307] ComputeBasicMatrix] End Path Computing [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.512 [mindspore/ccsrc/backend/common/somas/somas.cc:1309] ComputeBasicMatrix] Start Tensor Relation Computing [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.561 [mindspore/ccsrc/backend/common/somas/somas.cc:1462] ComputeMultiTensorConflicts] Start Computing Conflicts Pairs, tensors list size is 23 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.584 [mindspore/ccsrc/backend/common/somas/somas.cc:1469] ComputeMultiTensorConflicts] End Computing Conflicts Pairs (time taken 0ms) [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.595 [mindspore/ccsrc/backend/common/somas/somas.cc:1367] ComputeBasicMatrix] End Basic Conflict Computing (Bitset Model)(time taken 0ms) [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.614 [mindspore/ccsrc/backend/common/somas/somas.cc:178] Assign] End Computing Conflict Matrix [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.625 [mindspore/ccsrc/backend/common/somas/somas.cc:1533] Solve] Somas Assign start... [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.643 [mindspore/ccsrc/backend/common/somas/somas.cc:1555] Solve] Start Solving [INFO] PRE_ACT(164046,fffe87fff0f0,python):2024-01-10-11:41:05.112.757 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 20 tensors [INFO] PRE_ACT(164046,fffe877fe0f0,python):2024-01-10-11:41:05.112.790 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 20 tensors [INFO] PRE_ACT(164046,fffea17fa0f0,python):2024-01-10-11:41:05.112.780 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 20 tensors [INFO] PRE_ACT(164046,fffea0ff90f0,python):2024-01-10-11:41:05.112.771 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:230] Search] Calling FastSolver Search for 20 tensors [INFO] PRE_ACT(164046,fffe87fff0f0,python):2024-01-10-11:41:05.112.869 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 2/4 1205760 Bytes (0.00112295 GB) Shared Objects size(>), index(<) smallest [INFO] PRE_ACT(164046,fffe877fe0f0,python):2024-01-10-11:41:05.112.875 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 4/4 1205760 Bytes (0.00112295 GB) Single Object size(>), index(<) smallest [INFO] PRE_ACT(164046,fffea17fa0f0,python):2024-01-10-11:41:05.112.894 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 1/4 1205760 Bytes (0.00112295 GB) Shared Objects size(>), index(<) bestfit [INFO] PRE_ACT(164046,fffea0ff90f0,python):2024-01-10-11:41:05.112.909 [mindspore/ccsrc/backend/common/somas/somas_solver_core.cc:239] Search] 0 ms 3/4 1205760 Bytes (0.00112295 GB) Single Object size(>), index(<) bestfit [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.931 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:176] Solving] SOMAS SOLVER RESUME: [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.945 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:177] Solving] Best Solution:[1/4] [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.961 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:178] Solving] Best result:1205760 Bytes 0.00112295 GB (0.00112295 GB + 0 GB from lifelong tensors) [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.972 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:181] Solving] Best timing:0 ms [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.982 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:182] Solving] Best algorithm: Shared Objects [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.112.992 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:183] Solving] Best sorting strategy: size(>), index(<) [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.113.002 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:184] Solving] Best offset strategy: bestfit [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.113.013 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:185] Solving] Time elapsed: 0 ms [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.113.024 [mindspore/ccsrc/backend/common/somas/somas_solver_pre.cc:186] Solving] Spread:0 %% [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.113.061 [mindspore/ccsrc/backend/common/somas/somas.cc:1564] Solve] End Solving [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.113.085 [mindspore/ccsrc/backend/common/somas/somas.cc:2096] GenGraphStatisticInfo] Lower Bound: 1205760 (0.00112295 GB), Upper Bound: 2039296 (0.00189924 GB) [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.113.098 [mindspore/ccsrc/backend/common/somas/somas.cc:2099] GenGraphStatisticInfo] Total Dynamic Size (Upper Bound): 2039296 Theoretical Optimal Size (Lower Bound): 1205760 Total Workspace Size: 0 Total Communication Input Tensor Size: 0 Total Communication Output Tensor Size: 0 Total LifeLong All Tensor Size: 0 Total LifeLong Start Tensor Size: 0 Total LifeLong End Tensor Size: 2560 Reused Size(Allocate Size): 0 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.113.116 [mindspore/ccsrc/backend/common/somas/somas.cc:1583] Solve] Somas Assign end. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.113.154 [mindspore/ccsrc/backend/common/somas/somas.cc:380] UpdateSomasResultToGraph] Merged Block size: 3 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.113.167 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 0, offset: 602624, size: 602624 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.113.178 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 1, offset: 0, size: 602624 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.113.189 [mindspore/ccsrc/backend/common/somas/somas.cc:382] UpdateSomasResultToGraph] Merged Block: 2, offset: 1205248, size: 512 [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:05.113.199 [mindspore/ccsrc/backend/common/somas/somas.cc:189] Assign] Somas Allocate end. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:05.113.211 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ge_kernel_executor.cc:310] DoSomas] Somas allocate success for graph 3 somas size: 1205760 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.113.295 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:801] CreateDeviceAddress] Status record: start create device address. graph id: 3 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:05.113.565 [mindspore/ccsrc/runtime/device/device_address_utils.cc:454] CreateValueNodeDeviceAddress] No device address for value node:Default/data-9_6, debug name:ValueNode U [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:05.114.331 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/GetNext-op1_6, index is 1; cur kernel is Default/Reshape-op1_6, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:05.114.371 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/GetNext-op1_6, index is 1; cur kernel is Default/Reshape-op1_6, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:05.114.397 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2_6, index is 0; cur kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1_6, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:05.114.420 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/max_pool2d-MaxPool2d/MaxPool-op2_6, index is 0; cur kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/flatten-Flatten/Flatten-op1_6, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:05.114.439 [mindspore/ccsrc/runtime/device/device_address_utils.cc:757] UpdateDeviceAddress] Ref node pair: origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6, index is 0; cur kernel is Default/Reshape-op0_6, index is 0 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:05.114.460 [mindspore/ccsrc/runtime/device/device_address_utils.cc:797] UpdateDeviceAddress] Update device address: ref origin kernel is Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6, index is 0; cur kernel is Default/Reshape-op0_6, index is 0 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.114.477 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:808] CreateDeviceAddress] Status record: end create device address. graph id: 3 [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.114.552 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1129] CacheGraphOutputToFrontNodeWithIndex] Get graph backend output nodes. [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.114.604 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1137] CacheGraphOutputToFrontNodeWithIndex] Get graph front output nodes. [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.114.642 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1155] CacheGraphOutputToFrontNodeWithIndex] Backend output: Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/ReduceMean-op0_6 with index: 0 map to front node: Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits-op0_6 with index: 0 [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.114.655 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1155] CacheGraphOutputToFrontNodeWithIndex] Backend output: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6 with index: 0 map to front node: Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op0_6 with index: 0 [INFO] SESSION(164046,ffff82e63440,python):2024-01-10-11:41:05.114.668 [mindspore/ccsrc/backend/common/session/kernel_graph.cc:1155] CacheGraphOutputToFrontNodeWithIndex] Backend output: Default/GetNext-op1_6 with index: 1 map to front node: Default/GetNext-op0_6 with index: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.114.704 [mindspore/ccsrc/runtime/graph_scheduler/graph_compiler.cc:507] CompileGraph] Status record: end compile graph. graph id: 3 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.114.975 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:805] CompileGraphFromSegment] Compile cut segment, the cut node: @458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:CNode_571{[0]: ValueNode Return, [1]: 562} [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.115.128 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:521] Transform] Graph(kernel_graph_3) transforms actor begin, strategy:pipeline_with_execution_order [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.115.187 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:2619] PersistDeviceTensorForValueNode] The device address is not exist: ValueNode_572(U) [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.115.259 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1101] BuildDataSourceActor] Create queue data source actor: kernel_graph_3_DeviceDSActor_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.115.513 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1232] BuildLoopCountActor] Create loop count actor: kernel_graph_3_LoopCountActor [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.115.537 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1265] BuildOutputActor] Create output actor: kernel_graph_3_OutputActor [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.115.555 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1285] BuildDataPrepareActor] Create data prepare actor: kernel_graph_3_DataPrepareActor [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.115.612 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:866] CacheGraphOutputToActor] Cache graph 3 output node:Default/GetNext-op1_6 debug string:@kernel_graph_3:outputs{[0]: ValueNode GetNext} with index:1 to actor:kernel_graph_3_DeviceDSActor_3, from front node:Default/GetNext-op0_6 debug string:@458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:outputs{[0]: ValueNode GetNext} with index:1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.115.680 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:866] CacheGraphOutputToActor] Cache graph 3 output node:Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6 debug string:@kernel_graph_3:562{[0]: ValueNode PrimFunc_BiasAdd, [1]: 562, [2]: CNode_561, [3]: ValueNode 0} with index:0 to actor:Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6, from front node:Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op0_6 debug string:@458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:562{[0]: ValueNode PrimFunc_BiasAdd, [1]: 562, [2]: CNode_561, [3]: ValueNode 0} with index:0 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.115.710 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:809] AddSomasInfoForGraphOutput] The graph 3 output node:Default/network-TrainOneStepCell/network-WithLossCell/_backbone-LeNet5/fc3-Dense/BiasAdd-op1_6 with index: 0 somas enable or not: 1, somas offset: 545280, aligned size: 1536 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.115.785 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:866] CacheGraphOutputToActor] Cache graph 3 output node:Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/ReduceMean-op0_6 debug string:@kernel_graph_3:562{[0]: ValueNode PrimFunc_ReduceMean, [1]: 562, [2]: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), [3]: ValueNode false} with index:0 to actor:Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/ReduceMean-op0_6, from front node:Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits-op0_6 debug string:@458_382_mindspore_train_dataset_helper__DataWrapper_construct_560:562{[0]: ValueNode SparseSoftmaxCrossEntropyWithLogits, [1]: 562, [2]: 562} with index:0 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.115.801 [mindspore/ccsrc/runtime/graph_scheduler/scheduler_helper.cc:809] AddSomasInfoForGraphOutput] The graph 3 output node:Default/network-TrainOneStepCell/network-WithLossCell/_loss_fn-SoftmaxCrossEntropyWithLogits/ReduceMean-op0_6 with index: 0 somas enable or not: 1, somas offset: 546816, aligned size: 512 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.115.818 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1518] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_3 start. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.115.858 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:1520] LinkDataArrowInNonSinkMode] Get all u input of cnode in graph:kernel_graph_3 end. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.116.463 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 1_actor_set_kernel_graph_3_memory_actor_insert in 16.33 us [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.116.491 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 2_actor_set_kernel_graph_3_invalid_data_arrow_elimination in 2.27 us [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.116.565 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 3_actor_set_kernel_graph_3_multi_actor_fusion in 55.11 us [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.116.590 [mindspore/ccsrc/runtime/graph_scheduler/optimizer/optimizer.cc:54] Optimize] Run pass 4_actor_set_kernel_graph_3_batch_data_arrow_fusion in 5.73001 us [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.116.607 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:554] Transform] Graph(kernel_graph_3) transforms actor end. [WARNING] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.117.009 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:612] CompileGraphs] [PROF]compile_func_graph costs 32880 usec. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.117.057 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:615] CompileGraphs] Status record: end compile function graph: 458_382_mindspore_train_dataset_helper__DataWrapper_construct_560, produce actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.117.080 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end task_emit action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.117.101 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:268] SetLoopCount] Change vm_loop_flag to 0, set loop_size to 1 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.117.117 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1328] operator()] Status record: start execute action. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.117.137 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1330] operator()] Status record: end execute action. TotalTime = 0.213108, [19] [parse]: 0.00194856 [symbol_resolve]: 0.0915127, [1] [Cycle 1]: 0.0909233, [1] [resolve]: 0.0908999 [graph_reusing]: 7.333e-05 [meta_unpack_prepare]: 0.00078888 [pre_cconv]: 3.901e-05 [abstract_specialize]: 0.0645712 [pack_expand]: 0.00013866 [auto_monad]: 0.0010117 [inline]: 5.457e-05 [pre_auto_parallel]: 3.717e-05 [pipeline_split]: 3.617e-05 [optimize]: 0.0183163, [35] [py_interpret_to_execute]: 0.00022206 [rewriter_before_opt_a]: 0.00112723 [opt_a]: 0.0140631, [2] [Cycle 1]: 0.00997009, [30] [expand_dump_flag]: 1.686e-05 [switch_simplify]: 0.00033397 [a_1]: 0.0031484 [recompute_prepare]: 3.592e-05 [updatestate_depend_eliminate]: 0.00032973 [updatestate_assign_eliminate]: 4.524e-05 [updatestate_loads_eliminate]: 0.00021328 [parameter_eliminate]: 2.45e-06 [a_2]: 0.0006039 [accelerated_algorithm]: 3.021e-05 [pynative_shard]: 4.637e-05 [auto_parallel]: 4.04e-06 [parallel]: 3.483e-05 [merge_comm]: 1.637e-05 [allreduce_fusion]: 8.25999e-06 [virtual_dataset]: 1.947e-05 [get_grad_eliminate_]: 1.669e-05 [virtual_output]: 1.584e-05 [merge_forward]: 2.808e-05 [cell_reuse_recompute_pass]: 7.99999e-07 [cell_reuse_handle_not_recompute_node_pass]: 4.318e-05 [meta_fg_expand]: 2.563e-05 [after_resolve]: 2.504e-05 [a_after_grad]: 2.091e-05 [renormalize]: 0.00424805 [real_op_eliminate]: 2.589e-05 [auto_monad_grad]: 4.02e-06 [auto_monad_eliminator]: 0.00011369 [cse]: 0.00014122 [a_3]: 0.00013082 [Cycle 2]: 0.00132691, [30] [expand_dump_flag]: 1.48e-06 [switch_simplify]: 1.787e-05 [a_1]: 0.00023956 [recompute_prepare]: 1.509e-05 [updatestate_depend_eliminate]: 2.829e-05 [updatestate_assign_eliminate]: 2.562e-05 [updatestate_loads_eliminate]: 2.407e-05 [parameter_eliminate]: 1.41e-06 [a_2]: 0.00028345 [accelerated_algorithm]: 2.75e-05 [pynative_shard]: 4.504e-05 [auto_parallel]: 4.13e-06 [parallel]: 6.14001e-06 [merge_comm]: 1.018e-05 [allreduce_fusion]: 5.99e-06 [virtual_dataset]: 1.842e-05 [get_grad_eliminate_]: 1.583e-05 [virtual_output]: 1.586e-05 [merge_forward]: 2.247e-05 [cell_reuse_recompute_pass]: 4.60001e-07 [cell_reuse_handle_not_recompute_node_pass]: 4.035e-05 [meta_fg_expand]: 1.718e-05 [after_resolve]: 2.259e-05 [a_after_grad]: 1.979e-05 [renormalize]: 7.0002e-08 [real_op_eliminate]: 1.588e-05 [auto_monad_grad]: 1.81e-06 [auto_monad_eliminator]: 5.813e-05 [cse]: 9.025e-05 [a_3]: 0.00012458 [py_interpret_to_execute_after_opt_a]: 3.237e-05 [slice_cell_reuse_recomputed_activation]: 2.85e-06 [rewriter_after_opt_a]: 0.00078911 [convert_after_rewriter]: 3.222e-05 [order_py_execute_after_rewriter]: 2.102e-05 [opt_b]: 0.00067661, [1] [Cycle 1]: 0.00067099, [7] [b_1]: 0.00042001 [b_2]: 1.947e-05 [updatestate_depend_eliminate]: 2.232e-05 [updatestate_assign_eliminate]: 2.098e-05 [updatestate_loads_eliminate]: 2.321e-05 [renormalize]: 3.824e-05 [cse]: 8.692e-05 [cconv]: 2.677e-05 [opt_after_cconv]: 0.00030995, [1] [Cycle 1]: 0.00030473, [7] [c_1]: 6.024e-05 [parameter_eliminate]: 1.24e-06 [updatestate_depend_eliminate]: 2.381e-05 [updatestate_assign_eliminate]: 2.379e-05 [updatestate_loads_eliminate]: 2.327e-05 [cse]: 7.571e-05 [renormalize]: 6.363e-05 [remove_dup_value]: 9.412e-05 [tuple_transform]: 0.00021753, [1] [Cycle 1]: 0.00021225, [3] [d_1]: 0.00011952 [d_2]: 4.291e-05 [renormalize]: 3.3e-05 [add_cache_embedding]: 6.129e-05 [add_recomputation]: 0.000241 [cse_after_recomputation]: 8.826e-05, [1] [Cycle 1]: 8.258e-05, [1] [cse]: 7.66e-05 [environ_conv]: 2.355e-05 [label_micro_interleaved_index]: 2.42001e-06 [label_fine_grained_interleaved_index]: 2.11e-06 [assign_add_opt]: 3.039e-05 [slice_recompute_activation]: 2.21e-06 [micro_interleaved_order_control]: 1.85e-06 [full_micro_interleaved_order_control]: 1.66e-06 [comp_comm_scheduling]: 2.08e-06 [reorder_send_recv_between_fp_bp]: 1.86e-06 [comm_op_add_attrs]: 9.89996e-07 [add_comm_op_reuse_tag]: 2.119e-05 [overlap_opt_shard_in_pipeline]: 1.423e-05 [grouped_pairwise_exchange_alltoall]: 1.312e-05 [overlap_recompute_and_grad_model_parallel]: 2.01e-06 [overlap_grad_matmul_and_grad_allreduce]: 1.25e-05 [split_matmul_comm_elemetwise]: 1.373e-05 [split_layernorm_comm]: 1.71e-06 [process_send_recv_for_ge]: 8.90002e-07 [handle_group_info]: 7.80004e-07 [auto_monad_reorder]: 0.0001382 [get_jit_bprop_graph]: 2.594e-05 [eliminate_special_op_node]: 0.00063799 [validate]: 0.00013194 [distribtued_split]: 0.00018676 [task_emit]: 0.0331935 [execute]: 3.059e-05 Sums parse : 0.001949s : 0.93% symbol_resolve.resolve : 0.090900s : 43.52% graph_reusing : 0.000073s : 0.04% meta_unpack_prepare : 0.000789s : 0.38% pre_cconv : 0.000039s : 0.02% abstract_specialize : 0.064571s : 30.92% pack_expand : 0.000139s : 0.07% auto_monad : 0.001012s : 0.48% inline : 0.000055s : 0.03% pre_auto_parallel : 0.000037s : 0.02% pipeline_split : 0.000036s : 0.02% optimize.py_interpret_to_execute : 0.000222s : 0.11% optimize.rewriter_before_opt_a : 0.001127s : 0.54% optimize.opt_a.expand_dump_flag : 0.000018s : 0.01% optimize.opt_a.switch_simplify : 0.000352s : 0.17% optimize.opt_a.a_1 : 0.003388s : 1.62% optimize.opt_a.recompute_prepare : 0.000051s : 0.02% optimize.opt_a.updatestate_depend_eliminate : 0.000358s : 0.17% optimize.opt_a.updatestate_assign_eliminate : 0.000071s : 0.03% optimize.opt_a.updatestate_loads_eliminate : 0.000237s : 0.11% optimize.opt_a.parameter_eliminate : 0.000004s : 0.00% optimize.opt_a.a_2 : 0.000887s : 0.42% optimize.opt_a.accelerated_algorithm : 0.000058s : 0.03% optimize.opt_a.pynative_shard : 0.000091s : 0.04% optimize.opt_a.auto_parallel : 0.000008s : 0.00% optimize.opt_a.parallel : 0.000041s : 0.02% optimize.opt_a.merge_comm : 0.000027s : 0.01% optimize.opt_a.allreduce_fusion : 0.000014s : 0.01% optimize.opt_a.virtual_dataset : 0.000038s : 0.02% optimize.opt_a.get_grad_eliminate_ : 0.000033s : 0.02% optimize.opt_a.virtual_output : 0.000032s : 0.02% optimize.opt_a.merge_forward : 0.000051s : 0.02% optimize.opt_a.cell_reuse_recompute_pass : 0.000001s : 0.00% optimize.opt_a.cell_reuse_handle_not_recompute_node_pass : 0.000084s : 0.04% optimize.opt_a.meta_fg_expand : 0.000043s : 0.02% optimize.opt_a.after_resolve : 0.000048s : 0.02% optimize.opt_a.a_after_grad : 0.000041s : 0.02% optimize.opt_a.renormalize : 0.004248s : 2.03% optimize.opt_a.real_op_eliminate : 0.000042s : 0.02% optimize.opt_a.auto_monad_grad : 0.000006s : 0.00% optimize.opt_a.auto_monad_eliminator : 0.000172s : 0.08% optimize.opt_a.cse : 0.000231s : 0.11% optimize.opt_a.a_3 : 0.000255s : 0.12% optimize.py_interpret_to_execute_after_opt_a : 0.000032s : 0.02% optimize.slice_cell_reuse_recomputed_activation : 0.000003s : 0.00% optimize.rewriter_after_opt_a : 0.000789s : 0.38% optimize.convert_after_rewriter : 0.000032s : 0.02% optimize.order_py_execute_after_rewriter : 0.000021s : 0.01% optimize.opt_b.b_1 : 0.000420s : 0.20% optimize.opt_b.b_2 : 0.000019s : 0.01% optimize.opt_b.updatestate_depend_eliminate : 0.000022s : 0.01% optimize.opt_b.updatestate_assign_eliminate : 0.000021s : 0.01% optimize.opt_b.updatestate_loads_eliminate : 0.000023s : 0.01% optimize.opt_b.renormalize : 0.000038s : 0.02% optimize.opt_b.cse : 0.000087s : 0.04% optimize.cconv : 0.000027s : 0.01% optimize.opt_after_cconv.c_1 : 0.000060s : 0.03% optimize.opt_after_cconv.parameter_eliminate : 0.000001s : 0.00% optimize.opt_after_cconv.updatestate_depend_eliminate : 0.000024s : 0.01% optimize.opt_after_cconv.updatestate_assign_eliminate : 0.000024s : 0.01% optimize.opt_after_cconv.updatestate_loads_eliminate : 0.000023s : 0.01% optimize.opt_after_cconv.cse : 0.000076s : 0.04% optimize.opt_after_cconv.renormalize : 0.000064s : 0.03% optimize.remove_dup_value : 0.000094s : 0.05% optimize.tuple_transform.d_1 : 0.000120s : 0.06% optimize.tuple_transform.d_2 : 0.000043s : 0.02% optimize.tuple_transform.renormalize : 0.000033s : 0.02% optimize.add_cache_embedding : 0.000061s : 0.03% optimize.add_recomputation : 0.000241s : 0.12% optimize.cse_after_recomputation.cse : 0.000077s : 0.04% optimize.environ_conv : 0.000024s : 0.01% optimize.label_micro_interleaved_index : 0.000002s : 0.00% optimize.label_fine_grained_interleaved_index : 0.000002s : 0.00% optimize.assign_add_opt : 0.000030s : 0.01% optimize.slice_recompute_activation : 0.000002s : 0.00% optimize.micro_interleaved_order_control : 0.000002s : 0.00% optimize.full_micro_interleaved_order_control : 0.000002s : 0.00% optimize.comp_comm_scheduling : 0.000002s : 0.00% optimize.reorder_send_recv_between_fp_bp : 0.000002s : 0.00% optimize.comm_op_add_attrs : 0.000001s : 0.00% optimize.add_comm_op_reuse_tag : 0.000021s : 0.01% optimize.overlap_opt_shard_in_pipeline : 0.000014s : 0.01% optimize.grouped_pairwise_exchange_alltoall : 0.000013s : 0.01% optimize.overlap_recompute_and_grad_model_parallel : 0.000002s : 0.00% optimize.overlap_grad_matmul_and_grad_allreduce : 0.000013s : 0.01% optimize.split_matmul_comm_elemetwise : 0.000014s : 0.01% optimize.split_layernorm_comm : 0.000002s : 0.00% optimize.process_send_recv_for_ge : 0.000001s : 0.00% optimize.handle_group_info : 0.000001s : 0.00% auto_monad_reorder : 0.000138s : 0.07% get_jit_bprop_graph : 0.000026s : 0.01% eliminate_special_op_node : 0.000638s : 0.31% validate : 0.000132s : 0.06% distribtued_split : 0.000187s : 0.09% task_emit : 0.033194s : 15.89% execute : 0.000031s : 0.01% Time group info: ------[substitution.] 0.088931 1195 0.01% : 0.000006s : 5: substitution.depend_value_elim 0.01% : 0.000008s : 8: substitution.float_tuple_getitem_switch 97.92% : 0.087079s : 180: substitution.getattr_setattr_resolve 0.02% : 0.000019s : 40: substitution.graph_param_transform 1.49% : 0.001328s : 75: substitution.inline 0.01% : 0.000006s : 14: substitution.less_batch_normalization 0.01% : 0.000009s : 48: substitution.load_eliminater 0.24% : 0.000212s : 428: substitution.meta_unpack_prepare 0.01% : 0.000007s : 4: substitution.minmaximum_grad 0.00% : 0.000004s : 40: substitution.partial_unused_args_eliminate 0.01% : 0.000007s : 64: substitution.remove_not_recompute_node 0.01% : 0.000006s : 16: substitution.replace_old_param 0.02% : 0.000014s : 15: substitution.switch_simplify 0.03% : 0.000025s : 8: substitution.tuple_list_convert_item_index_to_positive 0.01% : 0.000010s : 8: substitution.tuple_list_get_item_const_eliminator 0.02% : 0.000016s : 8: substitution.tuple_list_get_item_depend_reorder 0.04% : 0.000039s : 15: substitution.tuple_list_get_item_eliminator 0.02% : 0.000015s : 8: substitution.tuple_list_get_set_item_eliminator 0.06% : 0.000049s : 104: substitution.updatestate_pure_node_eliminater 0.08% : 0.000073s : 107: substitution.updatestate_useless_node_eliminater ------[renormalize.] 0.004240 2 50.80% : 0.002154s : 1: renormalize.infer 49.20% : 0.002086s : 1: renormalize.specialize ------[replace.] 0.002444 256 0.48% : 0.000012s : 2: replace.depend_value_elim 75.95% : 0.001856s : 163: replace.getattr_setattr_resolve 18.14% : 0.000443s : 75: replace.inline 5.13% : 0.000125s : 15: replace.switch_simplify 0.30% : 0.000007s : 1: replace.tuple_list_get_item_eliminator ------[match.] 0.088182 256 0.00% : 0.000001s : 2: match.depend_value_elim 98.47% : 0.086836s : 163: match.getattr_setattr_resolve 1.51% : 0.001328s : 75: match.inline 0.02% : 0.000014s : 15: match.switch_simplify 0.00% : 0.000004s : 1: match.tuple_list_get_item_eliminator ------[func_graph_cloner_run.] 0.005158 106 68.03% : 0.003509s : 29: func_graph_cloner_run.FuncGraphClonerGraph 31.97% : 0.001649s : 77: func_graph_cloner_run.FuncGraphSpecializer ------[meta_graph.] 0.000000 0 ------[manager.] 0.000000 0 ------[pynative] 0.000000 0 ------[others.] 0.097553 122 1.22% : 0.001192s : 69: opt.transform.opt_a 0.42% : 0.000409s : 23: opt.transform.opt_b 93.17% : 0.090888s : 2: opt.transform.opt_resolve 0.73% : 0.000713s : 1: opt.transforms.meta_unpack_prepare 4.16% : 0.004062s : 20: opt.transforms.opt_a 0.06% : 0.000058s : 1: opt.transforms.opt_after_cconv 0.02% : 0.000018s : 1: opt.transforms.opt_b 0.16% : 0.000160s : 2: opt.transforms.opt_trans_graph 0.06% : 0.000056s : 3: opt.transforms.special_op_eliminate [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.117.753 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1385] Run] End [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.117.788 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:846] SaveCompiledGraph] Save compiled func graph(458_382_mindspore_train_dataset_helper__DataWrapper_construct_560) phase(eval.1704858064751680000.281469723781744.0)! [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.117.808 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:864] SaveCompiledGraph] End save compiled func graph! [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.117.821 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:942] CleanCompileRes] Clean compile resource start [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.120.198 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:956] CleanCompileRes] Clean compile resource end [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.120.224 [mindspore/ccsrc/pipeline/jit/ps/event_message_print.cc:37] PrintEventMessage] End compiling '_DataWrapper.construct'. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.120.239 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1039] CompileInner] Finish compiling. [WARNING] ME(164046:281472877868096,MainProcess):2024-01-10-11:41:05.120.960 [mindspore/parallel/_utils.py:259] You are suggested to use mindspore.context.set_auto_parallel_context(parameter_broadcast=True) or mindspore.common.set_seed() to share parameters among multi-devices. [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.122.370 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.122.414 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.122.441 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.122.618 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Float32, value=0), output index: 0 device address:0x49ef0b70 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.122.861 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[2], dtype=Int64, value=[32 10]), output index: 0 device address:0x49f80fd0 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.122.954 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode 0 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.123.040 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode 1 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.123.129 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[1], dtype=Int64, value=[32]), output index: 0 device address:0x4a95c4a0 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.123.211 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Int64, value=10), output index: 0 device address:0x49fbc990 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.123.296 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[], dtype=Float32, value=1), output index: 0 device address:0x4a966050 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.123.387 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode false [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.123.462 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:905] PrepareDataForSequenceAndScalarValue] Prepare device data for value node: ValueNode -1 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.123.542 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_prepare_actor.cc:733] PrepareDataForValueNodeTensor] Prepare device data for value node: ValueNode Tensor(shape=[1], dtype=Int64, value=[0]), output index: 0 device address:0x4de80a40 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:05.123.703 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] GE(164046,python):2024-01-10-11:41:05.215.832 [graph_var_manager.cc:1424][EVENT]167353 SetAllMemoryMaxValue:The graph_mem_max_size is 27917287424 and the var_mem_max_size is 5368709120 [INFO] GE(164046,python):2024-01-10-11:41:05.215.921 [graph_manager.cc:1248][EVENT]167353 PreRun:PreRun start: graph node size 2, session id 32, graph id 31, graph name online. [INFO] ATRACE(164046,python):2024-01-10-11:41:05.216.539 [atrace_api.c:28](tid:167353) AtraceCreate start [INFO] ATRACE(164046,python):2024-01-10-11:41:05.216.597 [trace_rb_log.c:84](tid:167353) [RUNTIME_ATRACE_DEV64_TS0] create ring buffer success, buffer size : 131152. [INFO] ATRACE(164046,python):2024-01-10-11:41:05.216.613 [atrace_api.c:32](tid:167353) AtraceCreate end [INFO] TDT(164046,python):2024-01-10-11:41:05.216.631 [client_manager.cpp:157][SetProfilingCallback][tid:167353] [TsdClient] set profiling callback success [INFO] GE(164046,python):2024-01-10-11:41:05.217.392 [parallel_partitioner.cc:165][EVENT]167353 DoPipelinePartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::PipelinePartition is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.217.434 [parallel_partitioner.cc:178][EVENT]167353 DoFlowGraphPartition:[GEPERFTRACE] The time cost of OptimizeSubgraph::FlowGraphPartition is [14] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.217.478 [graph_prepare.cc:1378][EVENT]167353 Init:[GEPERFTRACE] The time cost of FileConstantUtils::ConvertFileConstToConst is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.217.990 [graph_manager.cc:1050][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareInit is [527] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.218.023 [graph_manager.cc:1052][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.HandleSummaryOp is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.218.097 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ForToWhilePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.218.126 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::SavePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.218.185 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::NetOutputPass is [47] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.218.198 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of ProcessNetOutput::DataPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.218.246 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of CreateSubGraphWithScopePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.218.260 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubgraphMultiDimsClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.218.303 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of MultiBatchClonePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.218.403 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitVariableIntoSubgraphPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.218.422 [graph_manager.cc:1054][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.NormalizeGraph is [386] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.218.676 [graph_manager.cc:1055][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeGraphInit is [241] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.219.491 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:41:05.219.521 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.219.532 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.219.542 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [224] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.219.552 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [13] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.219.560 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [2] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:41:05.219.569 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [8] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.219.578 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [11] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.219.586 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [4] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.220.876 [graph_manager.cc:1056][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphForQuantize is [2179] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.220.940 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [5] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.220.959 [graph_prepare.cc:1982][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::ProcessBeforeInfershape is [48] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.221.283 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:41:05.221.307 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [0] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.318 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [0] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.331 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferShapePass is [141] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.342 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [7] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.351 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SplitShapeNPass is [0] micro second, call num is [4] [INFO] GE(164046,python):2024-01-10-11:41:05.221.360 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [3] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.381 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [6] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.393 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of InferValuePass is [3] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.435 [graph_prepare.cc:1983][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::FormatAndShapeProcess is [462] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.221.461 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::MarkForceUnknownForCondPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.221.475 [graph_prepare.cc:1984][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::CtrlFlowPreProcess is [23] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.221.490 [graph_prepare.cc:1985][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::multibatch::GetDynamicOutputShape is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.221.503 [graph_prepare.cc:1986][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::InsertAippOpUtil::Instance().UpdateDataNodeByAipp is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.221.514 [graph_prepare.cc:1987][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::SaveOriginalGraphToOmModel is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.221.528 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ShapeOperateOpRemovePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.221.543 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::ReplaceTransShapePass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.221.556 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::MarkAgnosticPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.221.629 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.644 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.654 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrintOpPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.666 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of NoUseReshapeRemovePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.677 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DropOutPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.709 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssertPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.725 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeRemovePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.737 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedConstPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.748 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of StopGradientPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.760 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreventGradientPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.768 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of PlaceholderWithDefaultPass is [0] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.776 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SnapshotPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.792 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of GuaranteeConstPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.801 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of VarIsInitializedOpPass is [5] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.810 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ParallelConcatStartOpPass is [0] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.818 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.221.840 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::PrunePass is [10] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.221.853 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareOptimize::HcclMemcpyPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.221.883 [graph_prepare.cc:1988][EVENT]167353 PrepareDynShape:[GEPERFTRACE] The time cost of Prepare::PrepareOptimize is [360] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.221.898 [graph_manager.cc:1065][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareDynShape is [989] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.234.093 [graph_manager.cc:1077][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraph is [12172] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.234.160 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PrepareRunningFormatRefiner::VariablePrepareOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.234.210 [graph_manager.cc:1080][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.preparer.PrepareRunningFormatRefiner is [80] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.236.857 [graph_manager.cc:1081][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeOriginalGraphJudgeInsert is [2631] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.236.900 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of SubexpressionMigrationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.236.927 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of UnusedArgsCleanPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.236.938 [graph_manager.cc:1082][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::SubexpressionMigration is [47] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.236.967 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeInputMemcpyPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.236.980 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SwitchDataEdgesBypass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.236.994 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::ConstantFuseSamePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.026 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CSEBeforeFuseDataNodesWithCommonInputPass is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.040 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::FuseDataNodesWithCommonInputPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.053 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::CommonSubexpressionEliminationPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.078 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::PermutePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.113 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::SameTransdataBreadthFusionPass is [25] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.134 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::VariableOpPass is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.159 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpWithoutReshapeFusionPass is [16] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.185 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::TransOpBreadthFusionPass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.203 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::DataFlowPreparePass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.215 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_1::MergeUnknownShapeNPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.224 [graph_manager.cc:2700][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_1 is [261] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.305 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of EnterPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.237.318 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AddNPass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.237.328 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchDeadBranchElimination is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.237.336 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of SwitchLogicRemovePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.237.345 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of MergePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.237.354 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CastRemovePass is [3] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.237.363 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransposeTransDataPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.237.371 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [3] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.237.380 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpSymmetryEliminationPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.237.388 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of TransOpNearbyAllreduceFusionPass is [3] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.237.397 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReplaceWithEmptyConstPass is [5] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.237.405 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionComputePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.237.414 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [7] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.237.422 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.237.437 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of UselessControlOutRemovePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.237.455 [graph_manager.cc:2741][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_2 is [214] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.466 [graph_manager.cc:2752][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of extern constant folding is [0] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.487 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::Migration is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.498 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ArgsClean is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.514 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::PrunePass is [6] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.530 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::NextIterationPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.542 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ControlTriggerPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.554 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MergeToStreamMergePass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.572 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SwitchToStreamSwitchPass is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.586 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::AttachStreamLabelPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.598 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::MultiBatchPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.609 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::SubgraphMultiDimsPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.627 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::IteratorOpPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.638 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::VariableRefUselessControlOutDeletePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.656 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::ReshapeRecoveryPass is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.669 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage1_3::RemoveSameConstPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.678 [graph_manager.cc:2810][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1_3 is [194] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.713 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of IdentityPass is [0] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.237.726 [graph_manager.cc:2821][EVENT]167353 OptimizeStage1:[GEPERFTRACE] The time cost of GraphPrepare::node_pass is [28] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.754 [graph_manager.cc:1087][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage1 is [798] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.889 [graph_manager.cc:1088][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeAfterStage1 is [122] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.925 [graph_manager.cc:1089][EVENT]167353 PreRunOptimizeOriginalGraph:[GEPERFTRACE] The time cost of GraphManager::GraphUtilsEx::InferShapeInNeed is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.950 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of PreRun::CtrlEdgeTransferPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.237.964 [graph_manager.cc:1097][EVENT]167353 PreRunOptimizeOriginalGraph:PreRun:PreRunOptimizeOriginalGraph success. [INFO] GE(164046,python):2024-01-10-11:41:05.237.988 [graph_manager.cc:3325][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::StagePartition is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.238.104 [engine_place.cc:144][EVENT]167353 Run:The time cost of DNN_VM_RTS_OP_STORE::CheckSupported is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.238.122 [engine_place.cc:144][EVENT]167353 Run:The time cost of aicpu_ascend_kernel::CheckSupported is [43] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.238.188 [graph_manager.cc:3351][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::GraphPartitionDynamicShape is [183] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.238.206 [graph_manager.cc:3364][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::CompositeEngine is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.238.274 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.238.294 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.238.420 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [113] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.238.446 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.238.483 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [27] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.238.512 [graph_manager.cc:3405][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::Partition1 is [291] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.238.529 [graph_manager.cc:3412][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPreProc is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.103 [graph_manager.cc:3422][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubGraph is [1560] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.140 [graph_manager.cc:3428][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::SetSubgraphPostProc is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.257 [graph_manager.cc:3467][EVENT]167353 SubgraphPartitionAndOptimization:[GEPERFTRACE] The time cost of OptimizeSubgraph::MergeSubGraph is [93] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.277 [graph_manager.cc:3377][EVENT]167353 OptimizeSubgraph:[GEPERFTRACE] The time cost of OptimizeSubgraph::SubgraphPartitionAndOptimization::AtomicEngine is [2057] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.295 [graph_manager.cc:1106][EVENT]167353 PreRunOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeSubgraph is [2313] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.307 [graph_manager.cc:1115][EVENT]167353 PreRunOptimizeSubGraph:PreRun:PreRunOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:41:05.240.329 [graph_manager.cc:1130][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.ReplacePrecompiledNodeWithOmGraph is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.372 [graph_manager.cc:1131][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::stages.optimizer.OptimizeWholeGraph is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.397 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::LinkGenMaskNodesPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.415 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::HcclContinuousMemcpyPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.425 [graph_manager.cc:2837][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses is [35] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.482 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ConstantFoldingPass is [9] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.240.494 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of ReshapeRemovePass is [2] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.240.503 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of CondRemovePass is [1] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.240.512 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of BitcastPass is [0] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.240.521 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of AssignRemovePass is [5] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.240.529 [base_pass.cc:339][EVENT]167353 Run:[GEPERFTRACE] The time cost of DimensionAdjustPass is [3] micro second, call num is [2] [INFO] GE(164046,python):2024-01-10-11:41:05.240.545 [graph_manager.cc:2864][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::MergedGraphNameToPasses is [98] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.557 [graph_manager.cc:2872][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::RemoveIsolatedConst is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.575 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::MultiBatchPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.588 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::RefIdentityDeleteOpPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.603 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::VariableRefDeleteOpPass is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.616 [compile_nodes_pass.cc:88][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.625 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::CompileNodesPass is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.637 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::SwapSpacePass is [2] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.706 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::InputOutputConnectionIdentifyPass is [60] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.729 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::AtomicAddrCleanPass is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.742 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::AfterMergePasses::EndOfSequenceAddControlPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.763 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::SubgraphPass is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.775 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize::AttachStreamLabelPass is [3] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.785 [graph_manager.cc:2927][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of OptimizeStage2::ControlAttrOptimize is [212] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.798 [graph_manager.cc:2937][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ModelBuilder::AssignFunctionalLabels is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.826 [graph_manager.cc:2943][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of MemcpyAddrAsyncPass::Run. is [19] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.240.840 [graph_manager.cc:2950][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of BufferPoolMemoryPass::Run. is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.241.021 [graph_manager.cc:2958][EVENT]167353 OptimizeStage2:[GEPERFTRACE] The time cost of ParallelGroupPass::Run. is [31] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.241.052 [graph_manager.cc:1132][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::OptimizeStage2 is [663] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.241.159 [graph_manager.cc:1135][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::GetCompilerStages(graph_node->GetGraphId()).optimizer.OptimizeGraphBeforeBuild is [92] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.241.200 [graph_manager.cc:2975][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of HandleMemoryRWConflict is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.241.305 [graph_manager.cc:2981][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of MemLayoutConflictOptimizer::Run. is [88] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.241.324 [pass_manager.cc:82][EVENT]167353 Run:[GEPERFTRACE] The time cost of OptimizeStage2::SetFftsPlusAttrPass is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.241.334 [graph_manager.cc:2986][EVENT]167353 MemConflictProc:[GEPERFTRACE] The time cost of SetFftsPlusAttrPass::last_passes.Run is [13] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.241.343 [graph_manager.cc:1136][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::MemConflictProc is [165] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.241.439 [graph_manager.cc:3555][EVENT]167353 Build:[GEPERFTRACE] The time cost of GraphManager::RecoverIrDefinitionAndModifyAippData is [70] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.241.496 [engine_partitioner.cc:1139][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionInitialize is [11] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.241.511 [engine_partitioner.cc:1142][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionMarkClusters is [1] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.241.598 [engine_partitioner.cc:1148][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSplitSubGraphs is [77] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.241.621 [engine_partitioner.cc:1155][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionSortSubGraphs is [9] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.241.654 [engine_partitioner.cc:1164][EVENT]167353 PartitionSubGraph:[GEPERFTRACE] The time cost of EnginePartitioner::PartitionAddPartitionsToGraphNode is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.241.682 [graph_builder.cc:865][EVENT]167353 SecondPartition:[GEPERFTRACE] The time cost of EnginePartitioner::Partition2 is [210] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.241.769 [graph_builder.cc:288][EVENT]167353 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::PreBuildModel is [56] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.241.910 [graph_builder.cc:293][EVENT]167353 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::CalcOpParam is [126] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.242.089 [model_builder.cc:1133][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignLogicalStreams is [88] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.242.344 [block_mem_assigner.cc:4069][EVENT]172230 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164046,python):2024-01-10-11:41:05.242.340 [block_mem_assigner.cc:4069][EVENT]172229 AssignMemoryWithReuse:Reuse memory open, memory_priority_mode is false. [INFO] GE(164046,python):2024-01-10-11:41:05.242.734 [graph_mem_assigner.cc:2166][EVENT]167353 SetInputOffset:[IMAS]AfterAssignMemory : online_31 memoffset[132096], memtype[2], theory_min[264192], zero_copy[132096], total_size[132096], no_reuse[132096], streams[1], topo_mode[DFS], mop[], io_reuse[0:0], alloc_mode[] [INFO] GE(164046,python):2024-01-10-11:41:05.242.821 [model_builder.cc:1144][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::AssignMemory is [709] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.242.845 [model_builder.cc:1152][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of SetInputOutputOffsetPass::Run is [8] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.242.860 [model_builder.cc:1157][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::CompileSingleOp is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.242.965 [model_builder.cc:1167][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::RefreshRealStream is [93] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.242.984 [model_builder.cc:1174][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::OptimizeStreamedWholeGraph is [5] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.243.006 [model_builder.cc:1180][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::MergeWeights is [7] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.243.039 [model_builder.cc:1184][EVENT]167353 BuildModelForGetTask:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelDef is [22] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.243.059 [graph_builder.cc:304][EVENT]167353 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::BuildModelForGetTask is [1128] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:41:05.243.242 [logger.cc:1071] 167353 ModelBindStream: model_id=64, stream_id=1345, flag=0. [INFO] GE(164046,python):2024-01-10-11:41:05.243.307 [task_generator.cc:804][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::SetStreamCtx is [4] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.243.367 [task_generator.cc:805][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::PrepareForGenerateTask is [46] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.243.872 [task_generator.cc:814][EVENT]167353 GenerateTask:[GEPERFTRACE] The time cost of TaskGenerator::DoGenerateTask is [487] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.243.889 [task_generator.cc:954][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::GenerateTask is [587] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.243.947 [task_generator.cc:967][EVENT]167353 GetTaskInfo:[GEPERFTRACE] The time cost of TaskGenerator::AddModelTaskToModel is [33] micro second. [INFO] RUNTIME(164046,python):2024-01-10-11:41:05.243.976 [logger.cc:1084] 167353 ModelUnbindStream: model_id=64, stream_id=1345, [INFO] GE(164046,python):2024-01-10-11:41:05.244.032 [graph_builder.cc:310][EVENT]167353 BuildForKnownShapeGraph:[GEPERFTRACE] The time cost of GraphBuilder::GetTaskInfo is [955] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.244.135 [graph_manager.cc:1152][EVENT]167353 PreRunAfterOptimizeSubGraph:[GEPERFTRACE] The time cost of GraphManager::Build is [2772] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.244.155 [graph_manager.cc:1164][EVENT]167353 PreRunAfterOptimizeSubGraph:PreRun:PreRunAfterOptimizeSubGraph success. [INFO] GE(164046,python):2024-01-10-11:41:05.244.185 [graph_manager.cc:1271][EVENT]167353 PreRun:[GEPERFTRACE] The time cost of FlowModelBuild is [26880] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.244.198 [graph_manager.cc:1272][EVENT]167353 PreRun:[GEPERFTRACE] GE PreRun End [INFO] ATRACE(164046,python):2024-01-10-11:41:05.244.503 [atrace_api.c:93](tid:167353) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:41:05.244.523 [atrace_api.c:95](tid:167353) AtraceDestroy end [INFO] GE(164046,python):2024-01-10-11:41:05.245.166 [model_introduction.cc:236][EVENT]167353 ConstructDynamicInfo:there is no case, no dynamic info [INFO] GE(164046,python):2024-01-10-11:41:05.245.190 [model_introduction.cc:294][EVENT]167353 ConstructNameOrder:there is no case, no data name order info. [INFO] GE(164046,python):2024-01-10-11:41:05.245.203 [model_introduction.cc:366][EVENT]167353 Data:model io_info size:116 [INFO] GE(164046,python):2024-01-10-11:41:05.248.606 [graph_converter.cc:838][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CreateMainNode is [1252] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.248.799 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [147] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.249.235 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [408] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.249.326 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [63] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.249.342 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [82] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.249.382 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [30] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.249.412 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [18] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.249.440 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of ZeroCopy is [17] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.249.506 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CEM is [55] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.249.568 [copy_flow_launch_fuse.cc:395][EVENT]167353 Run:[GEPERFTRACE] The time cost of Pass::CopyFlowLaunchFuse is [50] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.249.579 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of CopyFlowLaunch is [60] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.249.608 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of TrustOutTensor is [20] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.249.632 [base_optimizer.cc:70][EVENT]167353 Run:[GEPERFTRACE] The time cost of AicpuFuseHostInputs is [15] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.249.645 [graph_converter.cc:849][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::RunAllPass is [998] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.249.916 [graph_converter.cc:853][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::TopologicalSorting is [261] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.250.585 [graph_converter.cc:857][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::AppendGraphLevelData is [637] micro second. [INFO] GE(164046,python):2024-01-10-11:41:05.250.700 [graph_converter.cc:862][EVENT]167353 ConvertComputeGraphToExecuteGraph:[GEPERFTRACE] The time cost of ConvertComputeGraphToExecuteGraph::CalculatePriority is [87] micro second. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.255.485 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 1 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.255.623 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 1, execution time: 133.106 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.255.735 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.255.846 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.255.878 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.255.901 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.257.168 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.257.225 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.257.258 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.257.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.259.431 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 2 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.259.523 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 2, execution time: 2.20825 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.259.597 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.259.644 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.259.670 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.259.695 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.260.478 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.260.543 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.260.570 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.260.714 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:05.262.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.263.009 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 3, execution time: 2.38895 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.263.074 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.263.124 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.263.151 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.263.205 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.263.986 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.264.035 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.264.061 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.264.189 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:05.266.446 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 4 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.266.529 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 4, execution time: 2.41997 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.266.595 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.266.641 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.266.674 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.266.695 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.267.488 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.267.538 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.267.575 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.267.722 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:05.269.815 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 5 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.269.900 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 5, execution time: 2.27723 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.269.965 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.270.012 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.270.039 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.270.060 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.270.821 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.270.871 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.270.897 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.271.041 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.272.974 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 6 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.273.058 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 6, execution time: 2.11407 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.273.124 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.273.165 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.273.196 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.273.217 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.274.013 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.274.061 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.274.097 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.274.247 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:05.276.278 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 7 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.276.361 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 7, execution time: 2.21719 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.276.425 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.276.467 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.276.497 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.276.519 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.277.265 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.277.313 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.277.338 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.277.474 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:05.279.405 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 8 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.279.476 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 8, execution time: 2.09103 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.279.539 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.279.586 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.279.612 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.279.632 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.280.386 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.280.435 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.280.474 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.280.605 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:05.282.641 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 9 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.282.733 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 9, execution time: 2.21104 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.282.798 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.282.842 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.282.868 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.282.888 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.283.643 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.283.692 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.283.718 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.283.851 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:05.285.810 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 10 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.285.896 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 10, execution time: 2.12708 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.285.958 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.286.001 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.286.028 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.286.048 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.286.814 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.286.863 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.286.906 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.287.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:05.289.019 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 11 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.289.093 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 11, execution time: 2.13724 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.289.159 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.289.214 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.289.241 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.289.263 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.290.026 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.290.076 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.290.105 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.290.249 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:05.292.225 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 12 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.292.306 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 12, execution time: 2.1533 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.292.373 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.292.416 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.292.442 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.292.463 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.293.210 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.293.258 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.293.286 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.293.424 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.295.510 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 13 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.295.603 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 13, execution time: 2.25782 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.295.677 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.295.725 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.295.754 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.295.774 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.296.524 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.296.573 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.296.598 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.296.735 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:05.298.609 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 14 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.298.694 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 14, execution time: 2.04799 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.298.762 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.298.806 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.298.835 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.298.856 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.299.612 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.299.660 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.299.688 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.299.842 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:05.301.866 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 15 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.301.951 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 15, execution time: 2.20157 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.302.018 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.302.061 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.302.087 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.302.109 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.302.887 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.302.935 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.302.963 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.303.092 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:05.305.039 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 16 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.305.116 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 16, execution time: 2.10683 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.305.182 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.305.224 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.305.254 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.305.275 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.306.065 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.306.115 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.306.140 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.306.293 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:05.308.231 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 17 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.308.313 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 17, execution time: 2.1111 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.308.378 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.308.423 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.308.449 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.308.470 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.309.228 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.309.277 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.309.303 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.309.435 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.311.370 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 18 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.311.447 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 18, execution time: 2.09596 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.311.511 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.311.560 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.311.588 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.311.608 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.312.355 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.312.400 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.312.426 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.312.573 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:05.314.594 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 19 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.314.675 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 19, execution time: 2.19084 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.314.742 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.314.787 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.314.816 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.314.838 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.315.604 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.315.651 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.315.679 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.315.802 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:05.317.817 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 20 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.317.895 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 20, execution time: 2.16897 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.317.958 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.318.010 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.318.039 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.318.060 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.318.806 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.318.854 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.318.880 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.319.017 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:05.320.921 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 21 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.321.001 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 21, execution time: 2.07533 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.321.068 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.321.184 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.321.212 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.321.233 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.322.036 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.322.085 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.322.114 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.322.255 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:05.324.106 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 22 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.324.188 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 22, execution time: 2.02712 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.324.254 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.324.297 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.324.324 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.324.345 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.325.102 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.325.149 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.325.175 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.325.311 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:05.327.396 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 23 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.327.472 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 23, execution time: 2.25018 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.327.538 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.327.580 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.327.607 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.327.628 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.328.387 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.328.436 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.328.466 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.328.600 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.330.561 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 24 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.330.656 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 24, execution time: 2.14107 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.330.724 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.330.767 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.330.793 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.330.814 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.331.561 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.331.608 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.331.633 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.331.770 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.333.747 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 25 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.333.818 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 25, execution time: 2.13633 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.333.881 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.333.922 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.333.954 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.333.975 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.334.721 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.334.764 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.334.792 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.334.931 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.336.928 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 26 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.337.001 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 26, execution time: 2.1597 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.337.065 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.337.106 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.337.132 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.337.153 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.337.935 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.337.985 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.338.010 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.338.138 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:05.340.092 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 27 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.340.178 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 27, execution time: 2.11714 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.340.241 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.340.282 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.340.307 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.340.328 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.341.079 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.341.127 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.341.154 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.341.297 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:05.343.259 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 28 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.343.360 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 28, execution time: 2.1562 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.343.424 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.343.467 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.343.495 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.343.516 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.344.268 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.344.315 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.344.340 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.344.479 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.346.409 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 29 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.346.497 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 29, execution time: 2.11127 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.346.566 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.346.609 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.346.635 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.346.656 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.347.415 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.347.462 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.347.487 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.347.622 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:05.349.517 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 30 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.349.598 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 30, execution time: 2.06533 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.349.661 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.349.748 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.349.780 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.349.805 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.350.556 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.350.603 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.350.628 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.350.775 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:05.352.805 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 31 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.352.901 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 31, execution time: 2.22955 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.352.983 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.353.028 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.353.068 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.353.093 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.353.908 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.353.957 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.353.982 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.354.123 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:05.356.104 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 32 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.356.181 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 32, execution time: 2.15437 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.356.246 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.356.290 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.356.319 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.356.342 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.357.094 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.357.142 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.357.167 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.357.303 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:05.359.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 33 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.359.409 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 33, execution time: 2.19539 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.359.474 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.359.518 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.359.549 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.359.574 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.360.326 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.360.374 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.360.400 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:05.360.546 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:05.362.472 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 34 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.362.554 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 34, execution time: 2.10732 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.362.618 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.362.662 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.362.689 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.362.712 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.363.462 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.363.510 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.363.535 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.363.661 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe13fff0f0,python):2024-01-10-11:41:05.365.624 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 35 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.365.736 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 35, execution time: 2.116 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.365.803 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.365.846 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.365.874 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.365.897 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.366.767 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.366.815 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.366.841 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.366.981 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1bf4f0f0,python):2024-01-10-11:41:05.368.939 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 36 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.369.031 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 36, execution time: 2.14349 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.369.104 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.369.149 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.369.176 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.369.200 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.369.985 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.370.033 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.370.058 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.370.195 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:05.372.129 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 37 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.372.224 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 37, execution time: 2.11822 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.372.292 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.372.341 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.372.370 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.372.392 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.373.143 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.373.193 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.373.219 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.373.351 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:05.375.323 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 38 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.375.408 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 38, execution time: 2.14089 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.375.474 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.375.517 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.375.547 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.375.568 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.376.318 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.376.367 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.376.392 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.376.515 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1b74e0f0,python):2024-01-10-11:41:05.378.493 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 39 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.378.592 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 39, execution time: 2.15209 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.378.660 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.378.703 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.378.733 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.378.756 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.379.503 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.379.551 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.379.576 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,fffe1af4d0f0,python):2024-01-10-11:41:05.379.718 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,fffe1a74c0f0,python):2024-01-10-11:41:05.381.672 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 40 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.381.762 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 40, execution time: 2.13827 ms in multi thread or not: 1. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.381.828 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.381.870 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.381.900 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.381.922 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.382.675 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.382.724 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.382.750 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.382.880 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.384.891 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 41 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.384.990 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 41, execution time: 2.18939 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.385.058 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.385.105 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.385.132 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.385.152 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.385.952 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.386.000 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.386.026 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.386.128 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.388.030 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 42 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.388.114 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 42, execution time: 2.03934 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.388.183 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.388.230 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.388.258 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.388.279 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.389.037 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.389.085 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.389.111 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.389.210 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.391.126 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 43 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.391.212 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 43, execution time: 2.05127 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.391.293 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.391.339 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.391.367 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.391.388 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.392.153 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.392.200 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.392.227 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.392.327 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.394.256 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 44 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.394.341 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 44, execution time: 2.06344 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.394.411 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.394.457 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.394.485 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.394.506 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.395.282 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.395.328 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.395.354 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.395.451 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.397.354 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 45 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.397.439 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 45, execution time: 2.03859 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.397.520 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.397.570 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.397.597 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.397.618 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.398.407 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.398.457 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.398.485 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.398.586 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.400.481 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 46 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.400.565 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 46, execution time: 2.02912 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.400.633 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.400.682 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.400.711 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.400.732 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.401.484 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.401.532 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.401.558 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.401.655 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.403.607 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 47 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.403.691 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 47, execution time: 2.08456 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.403.772 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.403.819 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.403.849 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.403.871 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.404.624 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.404.670 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.404.697 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.404.794 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.406.716 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 48 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.406.801 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 48, execution time: 2.05619 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.406.872 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.406.920 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.406.948 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.406.969 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.407.730 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.407.777 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.407.803 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.407.901 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.409.843 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 49 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.409.931 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 49, execution time: 2.07935 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.410.018 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.410.067 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.410.097 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.410.119 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.410.879 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.410.926 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.410.952 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.411.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.412.928 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 50 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.413.011 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 50, execution time: 2.01092 ms in multi thread or not: 0. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.413.040 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:767] SetActorExecutionStrategy] Multi thread execution time cost: 2.14956 ms, single thread execution time cost: 2.06422 ms, decide to use multi thread execution or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.413.098 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.413.144 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.413.174 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.413.199 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.414.044 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.414.093 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.414.119 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.414.216 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.416.113 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 51 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.416.207 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 51, execution time: 2.03953 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.416.276 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.416.323 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.416.353 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.416.378 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.417.139 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.417.187 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.417.212 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.417.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.419.238 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 52 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.419.322 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 52, execution time: 2.06159 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.419.393 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.419.439 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.419.470 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.419.493 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.420.258 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.420.305 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.420.331 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.420.427 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.422.360 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 53 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.422.457 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 53, execution time: 2.07937 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.422.526 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.422.572 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.422.600 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.422.621 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.423.383 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.423.430 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.423.455 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.423.552 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.425.439 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 54 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.425.522 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 54, execution time: 2.01854 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.425.591 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.425.638 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.425.666 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.425.740 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.426.516 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.426.565 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.426.590 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.426.688 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.428.578 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 55 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.428.673 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 55, execution time: 2.03224 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.428.740 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.428.787 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.428.815 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.428.836 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.429.601 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.429.648 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.429.675 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.429.782 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.431.663 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 56 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.431.746 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 56, execution time: 2.01383 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.431.815 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.431.861 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.431.888 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.431.909 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.432.686 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.432.734 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.432.759 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.432.858 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.434.800 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 57 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.434.889 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 57, execution time: 2.08004 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.434.971 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.435.020 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.435.048 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.435.070 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.435.833 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.435.880 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.435.908 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.436.005 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.437.935 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 58 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.438.019 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 58, execution time: 2.06369 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.438.092 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.438.140 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.438.171 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.438.193 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.438.948 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.438.996 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.439.022 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.439.119 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.441.010 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 59 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.441.094 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 59, execution time: 2.02296 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.441.177 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.441.224 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.441.256 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.441.280 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.442.067 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.442.119 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.442.148 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.442.248 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.444.152 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 60 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.444.235 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 60, execution time: 2.03502 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.444.303 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.444.352 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.444.384 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.444.405 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.445.161 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.445.209 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.445.234 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.445.332 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.447.270 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 61 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.447.357 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 61, execution time: 2.07421 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.447.434 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.447.480 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.447.511 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.447.532 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.448.318 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.448.366 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.448.392 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.448.488 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.450.429 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 62 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.450.514 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 62, execution time: 2.07438 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.450.582 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.450.628 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.450.657 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.450.681 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.451.442 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.451.488 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.451.513 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.451.611 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.453.534 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 63 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.453.617 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 63, execution time: 2.05653 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.453.749 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.453.798 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.453.825 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.453.848 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.454.611 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.454.659 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.454.685 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.454.781 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.456.674 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 64 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.456.757 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 64, execution time: 2.02468 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.456.824 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.456.872 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.456.900 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.456.921 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.457.682 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.457.788 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.457.814 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.457.911 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.459.787 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 65 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.459.870 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 65, execution time: 2.00816 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.459.950 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.459.996 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.460.025 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.460.047 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.460.808 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.460.854 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.460.881 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.460.978 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.462.910 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 66 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.462.997 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 66, execution time: 2.06849 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.463.067 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.463.114 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.463.143 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.463.163 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.463.927 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.463.973 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.463.999 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.464.096 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.466.049 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 67 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.466.134 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 67, execution time: 2.08632 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.466.212 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.466.259 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.466.287 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.466.308 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.467.069 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.467.115 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.467.141 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.467.238 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.469.133 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 68 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.469.217 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 68, execution time: 2.02769 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.469.287 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.469.334 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.469.363 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.469.384 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.470.163 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.470.210 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.470.237 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.470.335 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.472.229 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 69 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.472.312 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 69, execution time: 2.02644 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.472.381 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.472.438 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.472.467 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.472.489 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.473.286 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.473.332 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.473.358 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.473.454 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.475.399 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 70 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.475.484 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 70, execution time: 2.07852 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.475.553 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.475.600 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.475.628 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.475.649 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.476.413 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.476.459 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.476.484 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.476.581 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.478.496 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 71 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.478.580 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 71, execution time: 2.0481 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.478.649 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.478.706 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.478.735 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.478.756 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.479.511 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.479.557 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.479.585 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.479.681 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.481.569 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 72 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.481.651 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 72, execution time: 2.01822 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.481.756 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.481.806 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.481.834 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.481.855 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.482.617 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.482.663 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.482.688 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.482.786 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.484.659 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 73 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.484.742 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 73, execution time: 2.00537 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.484.810 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.484.868 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.484.896 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.484.917 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.485.776 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.485.824 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.485.849 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.485.945 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.487.848 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 74 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.487.933 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 74, execution time: 2.03623 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.488.001 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.488.048 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.488.076 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.488.097 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.488.867 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.488.913 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.488.939 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.489.036 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.490.971 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 75 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.491.053 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 75, execution time: 2.06596 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.491.121 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.491.179 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.491.207 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.491.228 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.492.001 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.492.047 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.492.072 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.492.169 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.494.130 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 76 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.494.214 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 76, execution time: 2.0932 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.494.284 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.494.330 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.494.358 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.494.379 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.495.140 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.495.186 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.495.212 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.495.309 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.497.190 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 77 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.497.272 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 77, execution time: 2.01269 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.497.341 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.497.399 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.497.427 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.497.448 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] UTILS(164046,ffff82e63440,python):2024-01-10-11:41:05.498.235 [mindspore/ccsrc/utils/dynamic_obfuscation/registry_opaque_predicate.cc:112] init_calling_count] calling_count_ has been initialized to 0 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.498.283 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:1557] RunInner] VM loop size 1, loopsink size 1 [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.498.308 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1032] RunGraph] Status record: start run actor: kernel_graph_3 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.498.404 [mindspore/ccsrc/runtime/graph_scheduler/actor/data_source_actor.cc:39] FetchData] Data source actor(kernel_graph_3_DeviceDSActor_3) fetches data. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.500.311 [mindspore/ccsrc/runtime/graph_scheduler/actor/loop_count_actor.cc:56] IncreaseLoopCount] Loop count actor(kernel_graph_3_LoopCountActor) running, loop count: 1, current count: 1, total running count: 78 [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:05.500.396 [mindspore/ccsrc/runtime/graph_scheduler/graph_scheduler.cc:728] SetActorExecutionStrategy] kernel_graph_3 execution count: 78, execution time: 2.04107 ms in multi thread or not: 0. [INFO] VM(164046,ffff82e63440,python):2024-01-10-11:41:05.500.465 [mindspore/ccsrc/backend/graph_compiler/backend_base.cc:1062] RunGraph] Status record: end run actor: kernel_graph_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.500.511 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.500.538 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.500.560 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:381] BaseRefToPyDataWithUserData] user data is empty Accuracy: 0.07572115384615384 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.729.297 [mindspore/ccsrc/pipeline/jit/ps/init.cc:515] operator()] Start register... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.729.376 [mindspore/ccsrc/pipeline/jit/ps/init.cc:519] operator()] Start mindspore.profiler... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.729.455 [mindspore/ccsrc/pipeline/jit/ps/init.cc:527] operator()] Start EmbeddingCacheScheduler... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.729.479 [mindspore/ccsrc/pipeline/jit/ps/init.cc:534] operator()] Start releasing dataset handles... [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:05.729.538 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] MD(164046,ffff82e63440,python):2024-01-10-11:41:05.729.650 [mindspore/ccsrc/minddata/dataset/engine/python_runtime_context.cc:22] Terminate] Terminating a Dataset PythonRuntime. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.736.883 [mindspore/ccsrc/pipeline/jit/ps/init.cc:537] operator()] End release dataset handles. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:05.736.933 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2393] FinalizeCluster] Start finalize the cluster instance. [INFO] DISTRIBUTED(164046,ffff0134b0f0,python):2024-01-10-11:41:06.898.977 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:301] Heartbeat] The heartbeat thread is finished. [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:41:06.899.386 [mindspore/ccsrc/distributed/cluster/topology/compute_graph_node.cc:131] Finalize] The compute graph node has been unregistered successfully. [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:41:06.899.489 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:533] Finalize] Delete send event loop [INFO] DISTRIBUTED(164046,ffff02b4e0f0,python):2024-01-10-11:41:06.899.587 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:41:06.899.743 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:209] Finalize] Stop loop succ [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:41:06.899.762 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:540] Finalize] Delete recv event loop [INFO] DISTRIBUTED(164046,ffff0334f0f0,python):2024-01-10-11:41:06.899.823 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:41:06.899.957 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:209] Finalize] Stop loop succ [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:41:06.899.972 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:554] Finalize] Delete connection pool. [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:41:06.900.025 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:533] Finalize] Delete send event loop [INFO] DISTRIBUTED(164046,ffff01b4c0f0,python):2024-01-10-11:41:06.900.094 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:41:06.900.230 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:209] Finalize] Stop loop succ [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:41:06.900.247 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:540] Finalize] Delete recv event loop [INFO] DISTRIBUTED(164046,ffff0234d0f0,python):2024-01-10-11:41:06.900.288 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:82] EventLoopRun] Event epoll loop run end [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:41:06.900.407 [mindspore/ccsrc/distributed/rpc/tcp/event_loop.cc:209] Finalize] Stop loop succ [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:41:06.900.422 [mindspore/ccsrc/distributed/rpc/tcp/tcp_comm.cc:554] Finalize] Delete connection pool. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:06.900.440 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2396] FinalizeCluster] End finalize the cluster instance. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:06.900.454 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2335] ClearResAtexit] Pipeline clear all resource [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:06.900.555 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:290] RecordExitStatus] Status record: system exit. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:06.904.849 [mindspore/ccsrc/runtime/graph_scheduler/rpc_node_scheduler.cc:220] Clear] Start finalizing tcp server and client for rpc actors. [INFO] RUNTIME_FRAMEWORK(164046,ffff82e63440,python):2024-01-10-11:41:06.904.882 [mindspore/ccsrc/runtime/graph_scheduler/rpc_node_scheduler.cc:230] Clear] End finalizing tcp server and client for rpc actors. [INFO] ME(164046,ffff82e63440,python):2024-01-10-11:41:06.905.309 [mindspore/core/mindrt/src/actor/actormgr.cc:153] Finalize] mindrt Actors finish exiting. [INFO] ME(164046,ffff82e63440,python):2024-01-10-11:41:06.905.328 [mindspore/core/mindrt/src/actor/actormgr.cc:156] Finalize] mindrt Threads finish exiting. [INFO] ME(164046,ffff82e63440,python):2024-01-10-11:41:06.921.715 [mindspore/core/mindrt/src/actor/actormgr.cc:167] Finalize] mindrt IOMGRS finish exiting. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:06.922.495 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2207] ClearResPart1] Start Finalize StreamSynchronizer... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:06.922.532 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2209] ClearResPart1] End Finalize StreamSynchronizer... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:06.923.869 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:829] ClearRes] Clean executor resource! [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:06.923.898 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2223] ClearResPart2] Start clear PyNativeExecutor... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:06.924.233 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2225] ClearResPart2] End clear PyNativeExecutor. [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:41:06.924.283 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:179] ClearGraph] Remove all graphs in GraphManager [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:06.932.379 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2241] ClearResPart2] Start clear kernel runtime... [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:06.932.416 [mindspore/ccsrc/runtime/device/kernel_runtime_manager.cc:25] ClearRuntimeResource] Release device Ascend_3 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:06.932.432 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:240] ReleaseDeviceRes] Ascend finalize start [INFO] HCCL(164046,python):2024-01-10-11:41:06.932.538 [op_base.cc:1312][164046]com is not global com [INFO] HCCP(164046,python):2024-01-10-11:41:06.932.723 [ra_host.c:863]tid:164046,ra_socket_batch_close(863) : Input parameters: [0]th, phy_id[3], local_ip[3.0.0.0] [INFO] HCCP(164046,python):2024-01-10-11:41:06.932.970 [ra_host.c:1795]tid:164046,ra_socket_white_list_del(1795) : Input parameters: phy_id[3], local_ip[3.0.0.0], num[1] [INFO] HCCP(164046,python):2024-01-10-11:41:06.933.674 [ra_host.c:863]tid:164046,ra_socket_batch_close(863) : Input parameters: [0]th, phy_id[3], local_ip[3.0.0.0] [INFO] HCCP(164046,python):2024-01-10-11:41:06.960.312 [ra_host.c:863]tid:164046,ra_socket_batch_close(863) : Input parameters: [0]th, phy_id[3], local_ip[3.0.0.0] [INFO] HCCP(164046,python):2024-01-10-11:41:06.960.341 [ra_host.c:863]tid:164046,ra_socket_batch_close(863) : Input parameters: [1]th, phy_id[3], local_ip[3.0.0.0] [INFO] HCCP(164046,python):2024-01-10-11:41:06.960.352 [ra_host.c:863]tid:164046,ra_socket_batch_close(863) : Input parameters: [2]th, phy_id[3], local_ip[3.0.0.0] [INFO] HCCP(164046,python):2024-01-10-11:41:06.960.569 [ra_host.c:1795]tid:164046,ra_socket_white_list_del(1795) : Input parameters: phy_id[3], local_ip[3.0.0.0], num[1] [INFO] HCCP(164046,python):2024-01-10-11:41:06.964.216 [ra_host.c:941]tid:164046,ra_socket_listen_stop(941) : Input parameters: [0]th, phy_id[3], local_ip[3.0.0.0] [INFO] HCCP(164046,python):2024-01-10-11:41:06.964.353 [ra_host.c:525]tid:164046,ra_socket_deinit(525) : Input parameters: phy_id[3] family[2] local_ip[3.0.0.0] [INFO] HCCP(164046,python):2024-01-10-11:41:06.964.435 [ra_host.c:349]tid:164046,ra_deinit(349) : Input parameters: phy_id[3], nic_position:[1] [INFO] HCCP(164046,python):2024-01-10-11:41:06.964.447 [ra_hdc.c:1535]tid:164046,ra_hdc_deinit(1535) : hdc deinit start! phy_id[3] [INFO] HCCP(164046,python):2024-01-10-11:41:06.964.541 [ra_hdc.c:1570]tid:164046,ra_hdc_deinit(1570) : hdc deinit OK! phy_id[3] [INFO] ATRACE(164046,python):2024-01-10-11:41:06.964.635 [atrace_api.c:93](tid:164046) AtraceDestroy start [INFO] ATRACE(164046,python):2024-01-10-11:41:06.964.659 [atrace_api.c:95](tid:164046) AtraceDestroy end [INFO] HCCL(164046,python):2024-01-10-11:41:06.986.554 [op_base.cc:1332][164046]op_base comm destroy complete,take time [54045]us, rankNum[0], rank[4294967295], deviceLogicId[3] [INFO] HCCL_ADPT(164046,ffff82e63440,python):2024-01-10-11:41:06.986.619 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:260] FinalizeHccl] Start destroy hccl adapter for GRAPH_MODE [INFO] HCCL_ADPT(164046,ffff82e63440,python):2024-01-10-11:41:06.986.646 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:548] FinalizeHcclExec] Start finalize hccl exec. [INFO] HCCL(164046,python):2024-01-10-11:41:06.990.407 [hcom_executor.cc:43][164046][Finalize][HcomExecutor]Hcom Excutor Finalize end. ret[0] [INFO] HCCL_ADPT(164046,ffff82e63440,python):2024-01-10-11:41:06.990.453 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:556] FinalizeHcclExec] HcclExec destroy success [INFO] HCCL_ADPT(164046,ffff82e63440,python):2024-01-10-11:41:06.990.477 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:388] FinalizeKernelInfoStore] Start destroy hccl kernel info store. [INFO] HCCL_ADPT(164046,ffff82e63440,python):2024-01-10-11:41:06.990.519 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:410] FinalizeKernelInfoStore] Destroy hccl kernel info store success. [INFO] HCCL_ADPT(164046,ffff82e63440,python):2024-01-10-11:41:06.990.533 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:437] FinalizeHcclComm] Start finalize hccl comm. [INFO] HCCL_ADPT(164046,ffff82e63440,python):2024-01-10-11:41:06.990.578 [mindspore/ccsrc/plugin/device/ascend/hal/hccl_adapter/hccl_adapter.cc:273] FinalizeHccl] Destroy hccl adapter success. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:06.990.591 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:710] DestroyHccl] Hccl destroy successful. [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:06.990.642 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:783] operator()] Common mem pool info: Total allocated mem:1024M, peak used mem:4M, in used mem:0M, total idle mem:1023M. Block unit size:1024M, block counts:1, block[0] block size:1024M idle size:1023M [INFO] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:06.990.675 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:783] operator()] Persistent mem pool info: Total allocated mem:1024M, peak used mem:0M, in used mem:0M, total idle mem:1023M. Block unit size:1024M, block counts:1, block[0] block size:1024M idle size:1023M [WARNING] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:06.990.688 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:793] DumpDynamicMemPoolStateInfo] The dynamic memory pool total allocated mem:2048M, peak used mem:5M, in used mem:0M, total idle mem:71812935M, total eager free mem:0M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [WARNING] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:08.481.383 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:142] DeInitialize] Ascend Memory Adapter deinitialize success, statistics: Device HBM memory size: 32768M MindSpore Used memory size: 30678M MindSpore memory base address: 0x124100000000 Total Static Memory size: 2048M Total Dynamic memory size: 0M Dynamic memory size of this graph: 0M [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:08.502.708 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_kernel_runtime.cc:274] ReleaseDeviceRes] Ascend finalize end [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:08.502.762 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2243] ClearResPart2] End clear kernel runtime. [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:41:08.502.783 [mindspore/ccsrc/distributed/collective/collective_manager.cc:379] Finalize] Begin finalize collective manager. [INFO] DISTRIBUTED(164046,fffea1ffb0f0,python):2024-01-10-11:41:08.502.922 [mindspore/ccsrc/distributed/collective/collective_manager.cc:358] operator()] Start finalizing host communication lib. [INFO] DISTRIBUTED(164046,fffea1ffb0f0,python):2024-01-10-11:41:08.502.958 [mindspore/ccsrc/distributed/collective/collective_manager.cc:362] operator()] End finalizing host communication lib. [INFO] DISTRIBUTED(164046,fffea1ffb0f0,python):2024-01-10-11:41:08.502.970 [mindspore/ccsrc/distributed/collective/collective_manager.cc:367] operator()] Start finalizing device communication lib. [INFO] DISTRIBUTED(164046,fffea1ffb0f0,python):2024-01-10-11:41:08.502.981 [mindspore/ccsrc/distributed/collective/collective_manager.cc:371] operator()] End finalizing device communication lib. [INFO] DISTRIBUTED(164046,ffff82e63440,python):2024-01-10-11:41:08.503.019 [mindspore/ccsrc/distributed/collective/collective_manager.cc:386] Finalize] End finalize collective manager. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:08.503.036 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2258] ClearResPart2] Start clear device context... [INFO] ME(164046,ffff82e63440,python):2024-01-10-11:41:08.503.050 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:469] ClearDeviceContexts] Release device Ascend_3 [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:41:08.503.076 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:264] DeleteGraphRunner] Delete GraphRunner success [TRACE] GE(164046,python):2024-01-10-11:41:08.503.096 [status:INIT] [ge_api.cc:463]164046 ~Session:Start to destruct session. [TRACE] GE(164046,python):2024-01-10-11:41:08.503.122 [status:RUNNING] [ge_api.cc:475]164046 ~Session:Session id is 0 [TRACE] GE(164046,python):2024-01-10-11:41:08.503.132 [status:RUNNING] [ge_api.cc:476]164046 ~Session:Destroying session [TRACE] GE(164046,python):2024-01-10-11:41:08.503.988 [status:STOP] [ge_api.cc:491]164046 ~Session:Session Destructor finished [INFO] GE_ADPT(164046,ffff82e63440,python):2024-01-10-11:41:08.504.031 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:235] DeleteGeSession] Delete Ge Session success [TRACE] GE(164046,python):2024-01-10-11:41:08.504.055 [status:INIT] [ge_api.cc:301]164046 GEFinalize:GEFinalize start [INFO] GE(164046,python):2024-01-10-11:41:08.504.125 [execution_runtime.cc:80][EVENT]164046 FinalizeExecutionRuntime:Execution runtime finalize begin. [INFO] GE(164046,python):2024-01-10-11:41:08.504.143 [execution_runtime.cc:92][EVENT]164046 FinalizeExecutionRuntime:Execution runtime finalized. [TRACE] GE(164046,python):2024-01-10-11:41:08.504.153 [status:RUNNING] [ge_api.cc:313]164046 GEFinalize:Finalizing environment [INFO] TUNE(164046,python):2024-01-10-11:41:08.809.974 [cann_kb_pyfunc_mgr.cpp:127][CANNKB][Tid:164046]"CannKbPyfuncMgr: enter PyObjectDeinit function, reference_[1]" [INFO] TUNE(164046,python):2024-01-10-11:41:08.810.024 [cann_kb_pyfunc_mgr.cpp:138][CANNKB][Tid:164046]"CannKbPyfuncMgr: PyObjectDeinit function end successfully!" [INFO] GE(164046,python):2024-01-10-11:41:08.840.001 [gelib.cc:324][EVENT]164046 SystemFinalize:Online infer finalize GELib success. [TRACE] GE(164046,python):2024-01-10-11:41:09.112.651 [status:STOP] [ge_api.cc:341]164046 GEFinalize:GEFinalize finished [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:09.112.744 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:317] CloseTsd] Start to close tsd, ref = 1 [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:09.113.359 [mindspore/ccsrc/plugin/device/ascend/hal/device/tensorprint_utils.cc:449] DestroyTensorPrintThread] Succeed stop acl data channel for host queue [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:09.113.861 [mindspore/ccsrc/plugin/device/ascend/hal/device/tensorprint_utils.cc:375] JoinAclPrintThread] join acl tdt host receive process [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:09.113.924 [mindspore/ccsrc/plugin/device/ascend/hal/device/tensorprint_utils.cc:463] DestroyTensorPrintThread] Succeed destroy acl channel [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:09.113.953 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:119] DestoryHandler] The thread of ms_histogram_summary channel is being destroyed. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:09.113.969 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:115] ~MbufDataHandler] Channel ms_histogram_summary begins the destruction process. [INFO] DEVICE(164046,fffeb970d0f0,python):2024-01-10-11:41:09.298.246 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:67] ~ScopeAclTdtDataset] AcltdtDestroyDataset succeed. [INFO] DEVICE(164046,fffeba70f0f0,python):2024-01-10-11:41:09.298.394 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:67] ~ScopeAclTdtDataset] AcltdtDestroyDataset succeed. [INFO] DEVICE(164046,fffeb9f0e0f0,python):2024-01-10-11:41:09.298.414 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:67] ~ScopeAclTdtDataset] AcltdtDestroyDataset succeed. [INFO] DEVICE(164046,fffeb8f0c0f0,python):2024-01-10-11:41:09.298.464 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:67] ~ScopeAclTdtDataset] AcltdtDestroyDataset succeed. [INFO] DEVICE(164046,fffea3fff0f0,python):2024-01-10-11:41:09.298.497 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:67] ~ScopeAclTdtDataset] AcltdtDestroyDataset succeed. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:09.298.947 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:119] DestoryHandler] The thread of ms_scalar_summary channel is being destroyed. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:09.298.974 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:115] ~MbufDataHandler] Channel ms_scalar_summary begins the destruction process. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:09.299.224 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:119] DestoryHandler] The thread of ms_image_summary channel is being destroyed. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:09.299.238 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:115] ~MbufDataHandler] Channel ms_image_summary begins the destruction process. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:09.299.454 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:119] DestoryHandler] The thread of ms_tensor_summary channel is being destroyed. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:09.299.469 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:115] ~MbufDataHandler] Channel ms_tensor_summary begins the destruction process. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:09.299.685 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.h:119] DestoryHandler] The thread of ms_tensor_dump channel is being destroyed. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:09.299.700 [mindspore/ccsrc/plugin/device/ascend/hal/device/mbuf_receive_manager.cc:115] ~MbufDataHandler] Channel ms_tensor_dump begins the destruction process. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:09.299.966 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:337] CloseTsd] Call aclrtResetDevice, destroy and close tsd successful, ret[0] [INFO] ME(164046,ffff82e63440,python):2024-01-10-11:41:09.299.992 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:469] ClearDeviceContexts] Release device CPU_0 [INFO] ME(164046,ffff82e63440,python):2024-01-10-11:41:09.300.010 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:469] ClearDeviceContexts] Release device CPU_3 [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.050 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2260] ClearResPart2] End clear device context. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.062 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2262] ClearResPart2] Start clear AnalysisResultCacheMgr... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.076 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2264] ClearResPart2] End clear AnalysisResultCacheMgr. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.086 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2266] ClearResPart2] Start clear AnalysisContext... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.097 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2268] ClearResPart2] End clear AnalysisContext... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.107 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2270] ClearResPart2] Start clear AnalysisSchedule... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.254 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2272] ClearResPart2] End clear AnalysisSchedule... [INFO] DEBUG(164046,ffff82e63440,python):2024-01-10-11:41:09.300.272 [mindspore/ccsrc/debug/debugger/debugger.cc:305] Reset] Release Debugger resource. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.310 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2285] ClearResPart3] Start clear ClearObjectCache... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.322 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2287] ClearResPart3] End clear ClearObjectCache... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.333 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2289] ClearResPart3] Start clear Parser... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.345 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2291] ClearResPart3] End clear Parser... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.356 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2293] ClearResPart3] Start ClearTraceStack... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.368 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2295] ClearResPart3] End ClearTraceStack... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.378 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2297] ClearResPart3] Start clear InterpretNodeRecorder... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.389 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2299] ClearResPart3] End clear InterpretNodeRecorder... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.400 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2301] ClearResPart3] Start clear parallel::entire_costgraph... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.417 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2303] ClearResPart3] End clear parallel::entire_costgraph... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.427 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2305] ClearResPart3] Start clear ProtobufLibrary... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.706 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2307] ClearResPart3] End clear ProtobufLibrary... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.722 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2309] ClearResPart3] Start clear python_adapter... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.734 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2311] ClearResPart3] End clear python_adapter. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.745 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2315] ClearSingleton] Start clear singleton... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.826 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2331] ClearSingleton] End clear singleton. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.840 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2347] ClearResAtexit] Start unload dynamic lib... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.300.861 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2349] ClearResAtexit] End unload dynamic lib... [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.663.367 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:803] DelOneNetRes] Delete one net resource start [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.669.507 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:825] DelOneNetRes] Delete one net resource end. [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.669.616 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:803] DelOneNetRes] Delete one net resource start [INFO] PIPELINE(164046,ffff82e63440,python):2024-01-10-11:41:09.671.938 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:825] DelOneNetRes] Delete one net resource end. [WARNING] PRE_ACT(164046,ffff82e63440,python):2024-01-10-11:41:10.125.077 [mindspore/ccsrc/backend/common/mem_reuse/mem_dynamic_allocator.cc:793] DumpDynamicMemPoolStateInfo] The dynamic memory pool total allocated mem:2048M, peak used mem:5M, in used mem:0M, total idle mem:71812935M, total eager free mem:0M. Weight used size:0M, constant value used size:0M, kernel output used size:0M, other used size:0M. [INFO] DEVICE(164046,ffff82e63440,python):2024-01-10-11:41:10.125.192 [mindspore/ccsrc/plugin/device/ascend/hal/device/ascend_memory_adapter.cc:136] DeInitialize] DeInitialize Ascend Memory Adapter when it is not initialize corrupted size vs. prev_size IP address found on this node. Address info:{'family': 'inet', 'local': '127.0.0.1', 'prefixlen': 8, 'scope': 'host', 'label': 'lo', 'valid_life_time': 4294967295, 'preferred_life_time': 4294967295}. [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.264.190 [mindspore/ccsrc/pipeline/jit/ps/init.cc:515] operator()] Start register... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.264.255 [mindspore/ccsrc/pipeline/jit/ps/init.cc:519] operator()] Start mindspore.profiler... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.264.375 [mindspore/ccsrc/pipeline/jit/ps/init.cc:527] operator()] Start EmbeddingCacheScheduler... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.264.453 [mindspore/ccsrc/pipeline/jit/ps/init.cc:534] operator()] Start releasing dataset handles... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.264.513 [mindspore/ccsrc/pipeline/jit/ps/init.cc:537] operator()] End release dataset handles. [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.264.531 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2335] ClearResAtexit] Pipeline clear all resource [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.264.617 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:290] RecordExitStatus] Status record: system exit. [INFO] DEBUG(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.264.670 [mindspore/ccsrc/common/debug/env_config_parser.cc:152] ParseFromFile] The 'env_config_path' in 'mindspore.context.set_context(env_config_path={path})' is empty. [INFO] ME(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.264.753 [mindspore/core/mindrt/src/actor/actormgr.cc:153] Finalize] mindrt Actors finish exiting. [INFO] ME(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.264.767 [mindspore/core/mindrt/src/actor/actormgr.cc:156] Finalize] mindrt Threads finish exiting. [INFO] ME(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.264.780 [mindspore/core/mindrt/src/actor/actormgr.cc:167] Finalize] mindrt IOMGRS finish exiting. [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.264.819 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2207] ClearResPart1] Start Finalize StreamSynchronizer... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.264.846 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2209] ClearResPart1] End Finalize StreamSynchronizer... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.266.233 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:829] ClearRes] Clean executor resource! [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.266.269 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2223] ClearResPart2] Start clear PyNativeExecutor... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.266.420 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2225] ClearResPart2] End clear PyNativeExecutor. [INFO] GE_ADPT(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.266.479 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:179] ClearGraph] Remove all graphs in GraphManager [INFO] DEVICE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.275.465 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:385] UnregisterExternalAllocator] The graph_runner is not exist [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.275.500 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2241] ClearResPart2] Start clear kernel runtime... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.275.531 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2243] ClearResPart2] End clear kernel runtime. [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.275.548 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2258] ClearResPart2] Start clear device context... [INFO] ME(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.275.564 [mindspore/ccsrc/runtime/hardware/device_context_manager.cc:469] ClearDeviceContexts] Release device Ascend_0 [INFO] DEVICE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.041 [mindspore/ccsrc/plugin/device/ascend/hal/hardware/ascend_deprecated_interface.cc:317] CloseTsd] Start to close tsd, ref = 0 [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.074 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2260] ClearResPart2] End clear device context. [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.089 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2262] ClearResPart2] Start clear AnalysisResultCacheMgr... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.102 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2264] ClearResPart2] End clear AnalysisResultCacheMgr. [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.113 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2266] ClearResPart2] Start clear AnalysisContext... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.138 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2268] ClearResPart2] End clear AnalysisContext... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.149 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2270] ClearResPart2] Start clear AnalysisSchedule... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.445 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2272] ClearResPart2] End clear AnalysisSchedule... [INFO] DEBUG(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.482 [mindspore/ccsrc/debug/debugger/debugger.cc:101] Debugger] Debugger got device_target: Ascend [INFO] DEBUG(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.505 [mindspore/ccsrc/debug/debugger/debugger.cc:305] Reset] Release Debugger resource. [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.540 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2285] ClearResPart3] Start clear ClearObjectCache... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.552 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2287] ClearResPart3] End clear ClearObjectCache... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.563 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2289] ClearResPart3] Start clear Parser... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.578 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2291] ClearResPart3] End clear Parser... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.588 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2293] ClearResPart3] Start ClearTraceStack... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.613 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2295] ClearResPart3] End ClearTraceStack... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.624 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2297] ClearResPart3] Start clear InterpretNodeRecorder... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.638 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2299] ClearResPart3] End clear InterpretNodeRecorder... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.648 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2301] ClearResPart3] Start clear parallel::entire_costgraph... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.668 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2303] ClearResPart3] End clear parallel::entire_costgraph... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.679 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2305] ClearResPart3] Start clear ProtobufLibrary... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.980 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2307] ClearResPart3] End clear ProtobufLibrary... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.276.997 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2309] ClearResPart3] Start clear python_adapter... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.277.009 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2311] ClearResPart3] End clear python_adapter. [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.277.032 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2315] ClearSingleton] Start clear singleton... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.277.233 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2331] ClearSingleton] End clear singleton. [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.277.247 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2347] ClearResAtexit] Start unload dynamic lib... [INFO] PIPELINE(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.277.280 [mindspore/ccsrc/pipeline/jit/ps/pipeline.cc:2349] ClearResAtexit] End unload dynamic lib... Traceback (most recent call last): File "/home/jenkins/.local/bin/msrun", line 8, in sys.exit(main()) File "/home/jenkins/.local/lib/python3.7/site-packages/mindspore/parallel/cluster/run.py", line 129, in main run(args) File "/home/jenkins/.local/lib/python3.7/site-packages/mindspore/parallel/cluster/run.py", line 123, in run process_manager.run() File "/home/jenkins/.local/lib/python3.7/site-packages/mindspore/parallel/cluster/process_entity/_api.py", line 159, in run self.join_processes() File "/home/jenkins/.local/lib/python3.7/site-packages/mindspore/parallel/cluster/process_entity/_api.py", line 222, in join_processes raise RuntimeError("Distributed job exited with exception. Please check logs in " RuntimeError: Distributed job exited with exception. Please check logs in directory: . [INFO] GE_ADPT(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.860.120 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:261] DeleteGraphRunner] GraphRunner is not exist [INFO] GE_ADPT(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.860.200 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:225] DeleteGeSession] Ge Session is not exist [INFO] GE_ADPT(163896,ffff93db5010,python3.7):2024-01-10-11:41:13.860.215 [mindspore/ccsrc/transform/graph_ir/df_graph_manager.cc:179] ClearGraph] Remove all graphs in GraphManager [INFO] RUNTIME(163896,python3.7):2024-01-10-11:41:14.625.053 [runtime.cc:1737] 163896 ~Runtime: deconstruct runtime. F =================================== FAILURES =================================== __________________________________ test_msrun __________________________________ @pytest.mark.level0 @pytest.mark.platform_arm_ascend_training @pytest.mark.platform_x86_ascend_training @pytest.mark.env_single def test_msrun(): """ Feature: 'msrun' launch utility. Description: Launch distributed training job with dynamic cluster using msrun. Expectation: All workers are successfully spawned and running training. """ return_code = os.system( "export GLOG_v=1 && msrun --worker_num=4 --local_worker_num=4 --master_addr=127.0.0.1 "\ "--master_port=10969 --join=True "\ "test_msrun.py --device_target=Ascend --dataset_path=/home/workspace/mindspore_dataset/mnist" ) > assert return_code == 0 E assert 256 == 0 test_entry_msrun.py:34: AssertionError =========================== short test summary info ============================ FAILED test_entry_msrun.py::test_msrun - assert 256 == 0 ======================== 1 failed in 260.63s (0:04:20) =========================